Compare commits

...
Author SHA1 Message Date
jason.woltje 0ba50111f6 test(webui): the Triage drawer's no-Triage-request checks skip the static triage-settings.js script every page loads; it carries no data, and the Settings panel's own test shows a non-owner reads no route (#1873, lane triage-settings) 2026-11-18 15:06:03 -06:00
jason.woltje dab07f0ff5 fix(webui): Triage route review round: a save sends the revision its fields were read at, so a route saved elsewhere answers as stale instead of being written over, and an untouched page follows the store; a read follows the repo GET convention, so a LAN dev host with no Fetch Metadata reads the route while a write still needs one matching Origin and same-origin; a zone in another case is stored in its canonical spelling; focus lands on the saved line; the organization catalogues are read in parallel (#1873, lane triage-settings) 2026-11-18 14:40:22 -06:00
jason.woltje 561ff77e48 feat(webui): Settings > Triage reads and saves the Triage route (#1873, lane triage-settings)
GET and POST /api/triage/route, owner only behind the proxy-admin gate, with one
Origin and Sec-Fetch-Site same-origin on a write and Fetch Metadata on a read.
GET returns the stored route or says it is not set, plus the merged catalogue
choices (model keys, availability, compatible installed harnesses) with no
provider, upstream, module, key or path. POST takes model, harness, cap, time
zone and revision, checks the route with checkRoutes against the merged
catalogue (offered, available, compatible, installed), requires a known time
zone with no default, and writes through the store's writeRoute, so the lock,
the revision compare and the receipt are the store's. A stale revision answers
409 with the stored revision. updatedBy is the signed-in owner.

The Settings > Triage page shows only to the owner /api/config marks proxyAdmin.
Model and harness selects (the harness list is the chosen model's), a cap in
tokens, reset 00:00 read only, a time zone, Save. Not set before a first save;
a refusal is one line and keeps the fields; a stale save offers Reload route; a
second Save while one runs does nothing; one line says saving does not start
Triage. cli.mjs keeps route: null for the Triage host.
2026-11-18 14:40:21 -06:00
jason.woltje e1642be760 test(webui): drawer Stop review round: two Stops run at once in two drawers; coder's Stop the bus takes ends and repaints the page while ideas' still runs, and ideas keeps its running state; then ideas' is refused and each drawer keeps its own line (#1873, lane sp20-drawer-stop) 2026-11-18 14:33:52 -06:00
jason.woltje b23ccdfc12 fix(webui): Stop in the agent drawer: an archived or unavailable organization says so in Restart's plain line ending Nothing changed, not Outcome unknown, with one test each; each Stop keeps its own state per agent, as each restart does, so a second Stop or question in another drawer no longer drops a running Stop's line or its running state, with a two-drawer test (#1873, lane sp20-drawer-stop) 2026-11-18 14:33:52 -06:00
jason.woltje 57af33630d test(triage): a repeated usage.begin keeps the first baseline (review round, #1873, lane triage-cap) 2026-11-18 14:03:50 -06:00
jason.woltje 17b972b92d feat(triage): the host holds the daily token cap
Each turn's usage goes to <dataRoot>/triage/usage.jsonl, one line per
usage.end, 0600 and owned by the running user; a link, wrong mode or
owner, or a non-file refuses. The host holds the stored cap through
daily-quota mayLaunch and report before each start and each queued
message, lets a crossing turn finish, drains the queue with a host
line, sums the file again after a restart (a torn line counts as an
unknown turn), and GET /api/triage adds usageToday, cap and resetsAt.
With no route stored the fields say "not set". cli.mjs keeps route
null. (#1873, lane triage-cap)
2026-11-18 14:03:50 -06:00
jason.woltje d531d7181d fix(triage): review round for the triage key verbs
- The grace is the earliest retire time, not a timer: rotate, status and the
  README say the old key stays live until retire runs after it.
- A temp file left by a kill before the rename is removed under the lock by
  the next issue, rotate or retire; status counts it.
- The receipts file is checked under the lock before any change
  (triage-key-receipts-unsafe); a receipt that fails after the change is
  triage-key-unrecorded.
- A failed directory sync after the rename is triage-key-unsynced, after the
  receipt is written, not triage-key-unwritten.
- proxy-lock-busy points at mosaic proxy fence status.
- Tests: a sentinel lost on the read-back, FIFO key and receipts files, a stale
  lock reaped, a crash before the rename, the receipts checks, the sync
  failure; the no-op assertion is gone. (#1873)
2026-11-18 13:54:13 -06:00
jason.woltje 67e1ff193d feat(triage): mosaic triage key issue|rotate|retire|status, Triage's own model proxy key
issue adds one mst-<tag>-triage- key under the instance proxy lock (read,
sentinel, re-read, fence, PUT, read back, confirm) and writes it to
<configDir>/secrets/triage/model (directory 0700, file 0600, the running
user's; a link, a non-file, a second link, a wrong mode or owner refuses).
rotate adds a new key and keeps the old one until now plus the bus's
rotateGraceMinutes from proxy.json; retire removes retiring keys once due.
status reads the file and the list, takes no lock and writes nothing.
Outputs, errors and receipts carry key ids only. A crash at each step
leaves a state status names and the next verb clears. Org and no-org bus
reconciles leave the key alone. Nothing wires the key into the host (#1873)
2026-11-18 13:54:13 -06:00
25 changed files with 3658 additions and 49 deletions

No files matched your search

+70
View File
@@ -1452,3 +1452,73 @@ the model key against a catalogue yet.
The Console does not read this file: the Triage host keeps `route: null` until
the wiring slice.
## Triage key (#1873)
`mosaic triage key issue|rotate|retire|status` manage Triage's own model
proxy client key. Every verb refuses inside an agent run before anything is
read (exit 3), and takes no options.
The key is `mst-<tag>-triage-<43 base64url>`, where `<tag>` is the first 12
hex of sha256 of the config directory, the tag on the instance's own `msk-`
sentinel. An org bus reconciles only its `mso-<tag>-<org>-key-` names and a
no-org bus only its `msk-<tag>-` names, so neither removes it. The file is
`<configDir>/secrets/triage/model`: `{version, key, keyId, issuedAt,
rotateBy, retiring}`. The directory is 0700 and the file 0600, owned by the
running user. A link, a non-file, a second hard link, a wrong mode or owner, a
file over 64 KiB, or a record that does not match the shape refuses every verb
with `triage-key-unreadable` (exit 3).
- `issue` adds one key and writes the file. It refuses `triage-key-issued`
when a key is already issued; that is `rotate`.
- `rotate` adds a new key and moves the old one to `retiring`, with a retire
time of now plus `rotateGraceMinutes` from proxy.json, where the bus reads
it (default 120). `rotateBy` is 30 days on, as for an org key. An old key
the proxy no longer holds is dropped at once.
- `retire` removes retiring keys whose grace has passed or that the proxy no
longer holds. It changes nothing before then.
Nothing runs on a timer. The retire time is the earliest a retiring key may
go; the key stays live in the proxy until someone runs `retire` (or a later
`rotate` or `issue`) after it. `rotateBy` works the same way: `status` marks
it due, and only `rotate` replaces the key.
- `status` prints whether a key is issued, its key id, its age, its rotate-by
day, each retiring key, what the proxy list holds, a set fence, any key in
the Triage namespace the file does not hold, and the next command. It takes
no lock and writes nothing.
`issue`, `rotate` and `retire` hold the instance proxy lock that org buses
take (`<dataRoot>/proxy`). Each reads the list, refuses without this
instance's sentinel (`triage-key-proxy-open`), reads it again and refuses if
it moved (`triage-key-raced`), sets the fence, writes, reads back and clears
the fence. The proxy changes first and the file second. Each run that
changes something appends one line of key ids, never keys, to
`<dataRoot>/proxy/mosaic-keys.receipts.jsonl`. That file is checked under the
lock before anything changes; an unsafe one refuses
`triage-key-receipts-unsafe`. A receipt line that still fails after the change
refuses `triage-key-unrecorded`. No output, error or receipt holds a key.
The file is written to a 0600 temp file in the same directory, synced, renamed
into place, and the directory is synced. A failed directory sync after the
rename refuses `triage-key-unsynced`: the file is in place but may not survive
a power loss. `proxy-lock-busy` means another proxy writer holds the lock; if
none is running, `mosaic proxy fence status` shows the lock state.
After a crash:
- Killed after the proxy write and before the read-back: the fence stays set
and every proxy writer on the instance refuses `proxy-uncertain`. `status`
shows the fence. Restart the proxy, run `mosaic proxy fence clear
--proxy-restarted`, then run the verb again.
- Killed after the read-back and before the file write: the proxy holds a key
the file does not. `status` names it; the next `issue`, `rotate` or `retire`
removes it. A file that could not be written refuses
`triage-key-unwritten` and says to run the verb again.
- Killed after the temp file is written and before its rename: a
`.model.<pid>.<hex>.tmp` file that may hold a key stays in the triage
directory. `status` counts it; the next `issue`, `rotate` or `retire`
removes it under the lock.
- A `retire` killed after the proxy write leaves a retiring key the proxy no
longer holds. `status` marks it; the next `retire` drops it from the file.
Nothing gives the key to the Triage host yet; that is the wiring slice.
+392
View File
@@ -0,0 +1,392 @@
// #1873 triage-key: Triage's own model proxy client key, issued from the owner's terminal
// (design report reports/triage-route-design.md slice 3, Jason's ruling 2026-11-18).
//
// <configDir>/secrets/triage/model (directory 0700, file 0600, the running user's)
// { version, key, keyId, issuedAt, rotateBy, retiring: [{ key, keyId, issuedAt, retireAt }] }
//
// The key is `mst-<instance tag>-triage-<43 base64url>`, where the tag is the first 12 hex of
// sha256(configDir), the tag the instance's own `msk-` sentinel carries. An org bus reconciles only
// its own `mso-<tag>-<org>-key-` names and a no-org bus only `msk-<tag>-`, so neither touches it.
//
// Every proxy write runs under the instance proxy lock (packages/bus/src/proxy-lock.mjs), the lock
// org buses take: read the list, check this instance's sentinel, fence, PUT, read back, confirm.
// The proxy changes first and the file second, as the bus does for an agent key. A crash between
// leaves a key in the Triage namespace that the file does not hold; the next issue, rotate or retire
// removes it, and status names it. A crash after the PUT and before the read-back leaves the lock's
// uncertain fence, which every proxy writer on the instance refuses until a human clears it
// (`mosaic proxy fence clear --proxy-restarted`).
//
// Nothing runs on a timer. rotate records when the old key's grace ends; the key stays in the proxy
// until a later retire, rotate or issue removes it, and status says when that is due. The same goes
// for rotateBy: status marks it due; only rotate replaces the key.
//
// Nothing here prints, logs or records a key. Receipts and output carry key ids only (the first 12
// hex of sha256(key), the id the bus uses). Messages name files and options, never values.
// Nothing wires the key into the Triage host or the Console; that is a later slice.
import { closeSync, constants, fchmodSync, fstatSync, fsyncSync, lstatSync, mkdirSync, openSync, readdirSync, readFileSync, renameSync, unlinkSync, writeSync } from "node:fs";
import { createHash, randomBytes } from "node:crypto";
import { join } from "node:path";
import { isDeepStrictEqual } from "node:util";
import { createProxyLock } from "../../bus/src/proxy-lock.mjs";
import { loadManagementClient } from "../../model-proxy/src/client.mjs";
export const KEY_VERSION = 1;
// The org key lifetime and the rotate grace default. The bus holds both as literals
// (packages/bus/src/model-keys.mjs #issue: 30 days to rotateBy; packages/bus/src/runtime.mjs:
// `proxy.rotateGraceMinutes ?? 120`) and exports neither, and this lane may not change the bus.
// packages/cli/tests/triage-key.test.mjs drives the bus's own ModelKeys with the same clock and
// proves these give the same rotateBy and retireAt, so a change on either side goes red there.
// The grace itself is read from proxy.json first, where the bus reads it.
export const KEY_LIFETIME_DAYS = 30;
export const DEFAULT_ROTATE_GRACE_MINUTES = 120;
const FIELDS = Object.freeze(["version", "key", "keyId", "issuedAt", "rotateBy", "retiring"]);
const RETIRING_FIELDS = Object.freeze(["key", "keyId", "issuedAt", "retireAt"]);
const MAX_BYTES = 64 * 1024;
const MAX_RETIRING = 16;
const SECRET = /^[A-Za-z0-9_-]{43}$/;
const ISO = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/;
const DAY = /^\d{4}-\d{2}-\d{2}$/;
export class TriageKeyError extends Error {
constructor(code, message, exitCode = 3) {
super(message);
this.name = "TriageKeyError";
this.code = code;
this.exitCode = exitCode;
}
}
const unreadable = (why) => new TriageKeyError("triage-key-unreadable", `the Triage key store is unreadable: ${why}`);
export const keyIdOf = (key) => createHash("sha256").update(key).digest("hex").slice(0, 12);
export const instanceTag = (configDir) => createHash("sha256").update(configDir).digest("hex").slice(0, 12);
export const keyPrefix = (configDir) => `mst-${instanceTag(configDir)}-triage-`;
const sentinelPrefix = (configDir) => `msk-${instanceTag(configDir)}-sentinel-`;
const ownKey = (configDir) => (k) => typeof k === "string" && k.startsWith(keyPrefix(configDir)) && SECRET.test(k.slice(keyPrefix(configDir).length));
// Any name under the prefix is Triage's to remove, well formed or not, as the bus treats its own
// `msk-<tag>-` names. Only a well-formed key may be stored in the file.
const inNamespace = (configDir) => (k) => typeof k === "string" && k.startsWith(keyPrefix(configDir));
export function keyPaths(configDir) {
const secrets = join(configDir, "secrets");
const dir = join(secrets, "triage");
return { secrets, dir, file: join(dir, "model") };
}
const plain = (v) => v !== null && typeof v === "object" && !Array.isArray(v) && Object.getPrototypeOf(v) === Object.prototype;
const exactKeys = (v, keys) => plain(v) && Object.keys(v).length === keys.length && keys.every((k) => Object.hasOwn(v, k));
const canonicalTime = (s) => typeof s === "string" && ISO.test(s) && !Number.isNaN(Date.parse(s)) && new Date(s).toISOString() === s;
const canonicalDay = (s) => typeof s === "string" && DAY.test(s) && !Number.isNaN(Date.parse(`${s}T00:00:00Z`)) && new Date(`${s}T00:00:00Z`).toISOString().slice(0, 10) === s;
export function validRecord(doc, configDir) {
const own = ownKey(configDir);
return exactKeys(doc, FIELDS)
&& doc.version === KEY_VERSION
&& own(doc.key) && doc.keyId === keyIdOf(doc.key)
&& canonicalTime(doc.issuedAt) && canonicalDay(doc.rotateBy)
&& Array.isArray(doc.retiring) && doc.retiring.length <= MAX_RETIRING
&& doc.retiring.every((r) => exactKeys(r, RETIRING_FIELDS) && own(r.key) && r.key !== doc.key && r.keyId === keyIdOf(r.key)
&& canonicalTime(r.issuedAt) && canonicalTime(r.retireAt))
&& new Set(doc.retiring.map((r) => r.key)).size === doc.retiring.length;
}
const uidOf = (opts) => opts.uid ?? process.getuid();
// A real directory, the running user's, mode 0700. Missing: false, or made when `create` is on.
function checkDir(dir, uid, what, { create }) {
let st;
try { st = lstatSync(dir); }
catch (e) {
if (e.code !== "ENOENT") throw unreadable(`${what} cannot be read`);
if (!create) return false;
try { mkdirSync(dir, { mode: 0o700 }); } catch { throw unreadable(`${what} cannot be made`); }
try {
const fd = openSync(dir, constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW);
try { fchmodSync(fd, 0o700); } finally { closeSync(fd); }
} catch { throw unreadable(`${what} cannot be made`); }
st = lstatSync(dir);
}
if (st.isSymbolicLink()) throw unreadable(`${what} is a link`);
if (!st.isDirectory()) throw unreadable(`${what} is not a directory`);
if (st.uid !== uid) throw unreadable(`${what} is not owned by this user`);
if ((st.mode & 0o777) !== 0o700) throw unreadable(`${what} mode is not 0700`);
return true;
}
function checkFile(fd, uid, what, fail = unreadable) {
const st = fstatSync(fd);
if (!st.isFile()) throw fail(`${what} is not a regular file`);
if (st.nlink !== 1) throw fail(`${what} has more than one link`);
if (st.uid !== uid) throw fail(`${what} is not owned by this user`);
if ((st.mode & 0o777) !== 0o600) throw fail(`${what} mode is not 0600`);
return st;
}
// The stored record, or null when no key is issued. Throws triage-key-unreadable on anything unsafe.
// opts.uid (both directories and the file) and opts.fileUid (the file only) are test seams for the
// owner checks.
export function readKey(configDir, opts = {}) {
const { secrets, dir, file } = keyPaths(configDir);
const uid = uidOf(opts);
if (!checkDir(secrets, uid, "the secrets directory", { create: false })) return null;
if (!checkDir(dir, uid, "the triage secrets directory", { create: false })) return null;
let fd;
try { fd = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK); }
catch (e) {
if (e.code === "ENOENT") return null;
if (e.code === "ELOOP") throw unreadable("the key file is a link");
throw unreadable("the key file cannot be opened");
}
try {
const st = checkFile(fd, opts.fileUid ?? uid, "the key file");
if (st.size > MAX_BYTES) throw unreadable("the key file is too large");
let doc;
try { doc = JSON.parse(readFileSync(fd, "utf8")); } catch { throw unreadable("the key file is not JSON"); }
if (!validRecord(doc, configDir)) throw unreadable("the key file does not match the key shape");
return doc;
} finally { closeSync(fd); }
}
// Both directories, checked (and the triage one made) before any proxy write, so a store that cannot
// be written refuses while the proxy is still untouched.
function prepareDirs(configDir, uid) {
const { secrets, dir } = keyPaths(configDir);
if (!checkDir(secrets, uid, "the secrets directory", { create: false })) throw unreadable("the secrets directory is missing");
checkDir(dir, uid, "the triage secrets directory", { create: true });
}
// A temp file writeKey left when it was killed before its rename. It may hold a key.
const TEMP = /^\.model\.\d+\.[0-9a-f]{12}\.tmp$/;
const tempsIn = (dir) => { try { return readdirSync(dir).filter((n) => TEMP.test(n)); } catch { return []; } };
// Under the lock no other Triage writer runs, so every temp file left in the directory is dead.
function removeTemps(configDir) {
const { dir } = keyPaths(configDir);
for (const name of tempsIn(dir)) {
try { if (!lstatSync(join(dir, name)).isDirectory()) unlinkSync(join(dir, name)); }
catch { throw unreadable("a leftover temp file in the triage secrets directory cannot be removed"); }
}
}
// A directory sync that failed after the rename: the file is in place but may not be durable.
class NotSynced extends Error {}
// Atomic: a 0600 temp file in the same directory, fsync, rename, fsync of the directory.
function writeKey(configDir, record, uid, hook = () => {}) {
const { dir, file } = keyPaths(configDir);
prepareDirs(configDir, uid);
const temp = join(dir, `.model.${process.pid}.${randomBytes(6).toString("hex")}.tmp`);
let fd;
try { fd = openSync(temp, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600); }
catch { throw unreadable("the key file cannot be written"); }
try {
try { fchmodSync(fd, 0o600); writeSync(fd, `${JSON.stringify(record, null, 2)}\n`); fsyncSync(fd); }
finally { closeSync(fd); }
hook("rename");
renameSync(temp, file);
} catch { try { unlinkSync(temp); } catch {} throw unreadable("the key file cannot be written"); }
try {
hook("sync");
const dfd = openSync(dir, constants.O_RDONLY | constants.O_DIRECTORY);
try { fsyncSync(dfd); } finally { closeSync(dfd); }
} catch { throw new NotSynced(); }
}
// One line per write in the proxy lock's receipts file, ids only. `receiptsReady` runs under the
// lock before any change, so an unsafe receipts file refuses while nothing has changed yet.
const RECEIPTS_FLAGS = constants.O_WRONLY | constants.O_APPEND | constants.O_CREAT | constants.O_NOFOLLOW | constants.O_NONBLOCK;
const receiptsUnsafe = (why) => new TriageKeyError("triage-key-receipts-unsafe", `${why}; nothing was changed`);
function receiptsReady(stateDir, uid) {
let fd;
try { fd = openSync(join(stateDir, "mosaic-keys.receipts.jsonl"), RECEIPTS_FLAGS, 0o600); }
catch { throw receiptsUnsafe("the proxy receipts file cannot be opened"); }
try { checkFile(fd, uid, "the proxy receipts file", receiptsUnsafe); } finally { closeSync(fd); }
}
function receipt(stateDir, uid, entry) {
const unrecorded = new TriageKeyError("triage-key-unrecorded", "the change was made but its receipt line was not written; run mosaic triage key status");
let fd;
try { fd = openSync(join(stateDir, "mosaic-keys.receipts.jsonl"), RECEIPTS_FLAGS, 0o600); } catch { throw unrecorded; }
try {
checkFile(fd, uid, "the proxy receipts file", () => unrecorded);
writeSync(fd, `${JSON.stringify(entry)}\n`);
fsyncSync(fd);
} catch { throw unrecorded; }
finally { closeSync(fd); }
}
// The grace the bus uses: proxy.json's rotateGraceMinutes, or the bus default.
export const graceMinutes = (proxy) => proxy?.rotateGraceMinutes ?? DEFAULT_ROTATE_GRACE_MINUTES;
export const rotateByFor = (nowMs) => new Date(nowMs + KEY_LIFETIME_DAYS * 86400_000).toISOString().slice(0, 10);
export const retireAtFor = (nowMs, proxy) => new Date(nowMs + graceMinutes(proxy) * 60_000).toISOString();
const proxyFailure = (code) => {
const words = {
"proxy-lock-busy": "another proxy writer holds the instance proxy lock; try again when it ends, and if no writer is running, see mosaic proxy fence status",
"proxy-lock-uncertain": "the instance proxy lock is in a state this verb will not guess about; check <dataRoot>/proxy by hand",
"proxy-lock-invalid": "the instance proxy lock directory is missing or unsafe, or names another proxy",
"proxy-uncertain": "an earlier proxy write has an uncertain outcome; see mosaic proxy fence status",
"proxy-management-key-invalid": "the proxy management key file is missing or unsafe",
"proxy-client-invalid": "the proxy settings are not usable",
"proxy-client-down": "the proxy did not answer",
"proxy-client-refused": "the proxy refused the management request",
"proxy-client-redirect": "the proxy answered with a redirect",
"proxy-client-response": "the proxy gave an answer this verb cannot use",
};
return new TriageKeyError(code, words[code] ?? "the proxy write failed");
};
// opts.uid, opts.now and opts.hook(stage) are test seams; a stage hook may throw or kill the process.
function context({ configDir, system, uid, fileUid, now, hook }) {
if (!system?.proxy || !Number.isInteger(system.proxy.port)) throw new TriageKeyError("triage-key-no-proxy", "the instance has no model proxy set up (proxy.json)");
return {
configDir, dataRoot: system.dataRoot, proxy: system.proxy, stateDir: join(system.dataRoot, "proxy"),
uid: uid ?? process.getuid(), fileUid, now: now ?? Date.now, hook: hook ?? (() => {}),
};
}
function lockFor(c) {
try { return createProxyLock({ stateDir: c.stateDir, identity: `${c.configDir}:${c.proxy.port}` }); }
catch (e) { throw proxyFailure(e.code ?? "proxy-lock-invalid"); }
}
async function clientFor(c) {
try { return await loadManagementClient({ configDir: c.configDir, proxy: c.proxy }); }
catch (e) { throw proxyFailure(e.code ?? "proxy-management-key-invalid"); }
}
function checkSentinel(c, keys) {
const prefix = sentinelPrefix(c.configDir);
if (!keys.some((k) => k.startsWith(prefix) && SECRET.test(k.slice(prefix.length)))) {
throw new TriageKeyError("triage-key-proxy-open", "the proxy's key list has no sentinel for this instance, so it may be another proxy");
}
}
// Under the instance proxy lock: read the list, check the sentinel, run `plan`, write the result.
// `plan(keys, record)` returns { after, record, removed, added, done }; `record` null leaves the
// file alone. The proxy changes first, then the file, then one receipt line.
async function change(c, op, plan) {
const lock = lockFor(c);
const client = await clientFor(c);
try {
return await lock.run(async () => {
const before = await client.getKeys();
checkSentinel(c, before);
const current = readKey(c.configDir, { uid: c.uid, fileUid: c.fileUid });
const result = plan(before, current);
if (result.record) prepareDirs(c.configDir, c.uid);
removeTemps(c.configDir);
receiptsReady(c.stateDir, c.uid);
const after = result.after;
const removed = before.filter((k) => !after.includes(k));
const added = after.filter((k) => !before.includes(k));
// A key in another namespace never leaves the list.
if (removed.some((k) => !inNamespace(c.configDir)(k))) throw new TriageKeyError("triage-key-failed", "the change would remove a key outside the Triage namespace");
if (added.length || removed.length) {
// As the bus does: a list that moved since the first read is not overwritten.
if (!isDeepStrictEqual(await client.getKeys(), before)) throw new TriageKeyError("triage-key-raced", "the proxy's key list changed while this verb ran; run it again");
lock.fence({ org: null, op: `triage.${op}` });
await client.putKeys(after);
c.hook("put");
const confirmed = await client.getKeys();
if (!confirmed.some((k) => k.startsWith(sentinelPrefix(c.configDir))) || added.some((k) => !confirmed.includes(k)) || removed.some((k) => confirmed.includes(k))) {
throw new TriageKeyError("triage-key-unconfirmed", "the proxy did not confirm the change; the instance proxy fence stays set, see mosaic proxy fence status");
}
lock.confirm();
}
c.hook("write");
let unsynced = false;
if (result.record) {
try { writeKey(c.configDir, result.record, c.uid, c.hook); }
catch (e) {
if (e instanceof NotSynced) unsynced = true;
else throw new TriageKeyError("triage-key-unwritten", `the proxy list changed but ${keyPaths(c.configDir).file} was not written (${e.message}); fix the file's directory, then run mosaic triage key ${op} again, which clears what this run left`);
}
}
c.hook("receipt");
// A run that changed neither the proxy nor the file leaves no receipt.
if (added.length || removed.length || result.record) receipt(c.stateDir, c.uid, { op: `triage.key.${op}`, time: new Date(c.now()).toISOString(), addedKeyIds: added.map(keyIdOf), removedKeyIds: removed.map(keyIdOf) });
// The file is in place and the receipt written; only its durability is in doubt.
if (unsynced) throw new TriageKeyError("triage-key-unsynced", `${keyPaths(c.configDir).file} was written but its directory could not be synced, so it may not survive a power loss; run mosaic triage key status`);
return { ...result.done, addedKeyIds: added.map(keyIdOf), removedKeyIds: removed.map(keyIdOf) };
});
} catch (e) {
if (e instanceof TriageKeyError) throw e;
throw proxyFailure(typeof e?.code === "string" && /^proxy-[a-z-]+$/.test(e.code) ? e.code : "triage-key-failed");
}
}
const newKey = (configDir) => keyPrefix(configDir) + randomBytes(32).toString("base64url");
// Keys in the Triage namespace that the file does not hold: left by a crash between the PUT and the
// file write, or by a file that was removed by hand.
const orphansOf = (configDir, keys, record) => {
const held = record ? [record.key, ...record.retiring.map((r) => r.key)] : [];
return keys.filter((k) => inNamespace(configDir)(k) && !held.includes(k));
};
export async function issueKey(opts) {
const c = context(opts);
return change(c, "issue", (before, current) => {
if (current) throw new TriageKeyError("triage-key-issued", "a Triage key is already issued; use mosaic triage key rotate");
const orphans = orphansOf(c.configDir, before, null);
const key = newKey(c.configDir), nowMs = c.now();
const record = { version: KEY_VERSION, key, keyId: keyIdOf(key), issuedAt: new Date(nowMs).toISOString(), rotateBy: rotateByFor(nowMs), retiring: [] };
return { after: [...before.filter((k) => !orphans.includes(k)), key], record, done: { keyId: record.keyId, rotateBy: record.rotateBy } };
});
}
// Retiring keys whose grace has passed, or that the proxy no longer holds.
const dueOf = (record, keys, nowMs) => record.retiring.filter((r) => Date.parse(r.retireAt) <= nowMs || !keys.includes(r.key));
export async function rotateKey(opts) {
const c = context(opts);
return change(c, "rotate", (before, current) => {
if (!current) throw new TriageKeyError("triage-key-not-issued", "no Triage key is issued; use mosaic triage key issue");
const nowMs = c.now();
const orphans = orphansOf(c.configDir, before, current);
const due = dueOf(current, before, nowMs);
const keep = before.includes(current.key);
const retiring = [...current.retiring.filter((r) => !due.includes(r)),
...(keep ? [{ key: current.key, keyId: current.keyId, issuedAt: current.issuedAt, retireAt: retireAtFor(nowMs, c.proxy) }] : [])];
if (retiring.length > MAX_RETIRING) throw new TriageKeyError("triage-key-failed", "too many keys are retiring; run mosaic triage key retire once their grace has passed");
const key = newKey(c.configDir);
const record = { version: KEY_VERSION, key, keyId: keyIdOf(key), issuedAt: new Date(nowMs).toISOString(), rotateBy: rotateByFor(nowMs), retiring };
const drop = [...orphans, ...due.map((r) => r.key)];
return {
after: [...before.filter((k) => !drop.includes(k)), key], record,
done: { keyId: record.keyId, rotateBy: record.rotateBy, old: keep ? { keyId: current.keyId, retireAt: retiring.at(-1).retireAt } : { keyId: current.keyId, lost: true } },
};
});
}
export async function retireKeys(opts) {
const c = context(opts);
return change(c, "retire", (before, current) => {
const nowMs = c.now();
const orphans = orphansOf(c.configDir, before, current);
const due = current ? dueOf(current, before, nowMs) : [];
const record = current && due.length ? { ...current, retiring: current.retiring.filter((r) => !due.includes(r)) } : null;
const drop = [...orphans, ...due.map((r) => r.key)];
return { after: before.filter((k) => !drop.includes(k)), record, done: { droppedKeyIds: due.map((r) => r.keyId) } };
});
}
// Read only: the file, and when the proxy is set up, what its list holds. No lock, no write.
// The fence is checked by name only, so status writes nothing, not even the lock's identity file.
export async function keyStatus(opts) {
const { configDir, system, uid, fileUid } = opts;
const record = readKey(configDir, { uid, fileUid });
const proxy = { state: "not-set-up", fence: false, leftoverTemps: tempsIn(keyPaths(configDir).dir).length };
if (system?.proxy && Number.isInteger(system.proxy.port)) {
try { lstatSync(join(system.dataRoot, "proxy", "mosaic-keys.uncertain")); proxy.fence = true; }
catch (e) { if (e.code !== "ENOENT") proxy.fence = true; }
try {
const client = await loadManagementClient({ configDir, proxy: system.proxy });
const keys = await client.getKeys();
const prefix = sentinelPrefix(configDir);
proxy.state = keys.some((k) => k.startsWith(prefix) && SECRET.test(k.slice(prefix.length))) ? "read" : "no-sentinel";
proxy.holdsKey = record ? keys.includes(record.key) : null;
proxy.retiringHeld = record ? record.retiring.map((r) => keys.includes(r.key)) : [];
proxy.orphanKeyIds = orphansOf(configDir, keys, record).map(keyIdOf);
} catch (e) { proxy.state = "unreadable"; proxy.code = typeof e?.code === "string" && /^proxy-[a-z-]+$/.test(e.code) ? e.code : "proxy-client-response"; }
}
return { record, proxy };
}
+88 -1
View File
@@ -13,15 +13,23 @@
// Nothing here wires the route into the Triage host: the Console keeps `route: null` until the
// wiring slice. `set` checks the harness against the repository's harness modules; it does not
// check the model key against a catalogue, since none exists outside an organization's bus yet.
//
// mosaic triage key issue | rotate | retire | status
//
// Triage's own model proxy client key (packages/cli/src/triage-key.mjs). Human verbs as well,
// refused in an agent run before anything is read. No output names a key, only its id.
import { lstatSync } from "node:fs";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { readRoute, RouteStoreError, writeRoute } from "../../webui/src/triage-route-store.mjs";
import { loadSystem } from "./config.mjs";
import { configDir } from "../../organization/src/organization-config.mjs";
import { issueKey, keyPaths, keyStatus, retireKeys, rotateKey, TriageKeyError } from "./triage-key.mjs";
import { CliError } from "./errors.mjs";
import { refuseInsideAgent } from "./transport.mjs";
export const TRIAGE_USAGE = "mosaic triage route show | set --model KEY --harness NAME --cap TOKENS --time-zone ZONE [--revision N]";
export const TRIAGE_USAGE = "mosaic triage route show | set --model KEY --harness NAME --cap TOKENS --time-zone ZONE [--revision N] | key issue | rotate | retire | status";
const KEY_VERBS = ["issue", "rotate", "retire", "status"];
const HARNESS_ROOT = fileURLToPath(new URL("../../plugins/harness/", import.meta.url));
const VALUES = ["--model", "--harness", "--cap", "--time-zone", "--revision"];
const WHOLE = /^(0|[1-9][0-9]*)$/;
@@ -73,10 +81,89 @@ function show(route) {
].join("\n");
}
// A key error prints its code and fixed words. Anything else becomes one fixed line, so no value
// that an unexpected error might carry reaches the terminal.
function keyError(error) {
if (error instanceof CliError) return error;
if (error instanceof TriageKeyError) return new CliError(`${error.code}: ${error.message}`, error.exitCode);
return new CliError("triage-key-failed: the key verb stopped on an unexpected error; run mosaic triage key status", 3);
}
const age = (ms) => {
const m = Math.max(0, Math.floor(ms / 60_000));
return `${Math.floor(m / 1440)}d ${Math.floor((m % 1440) / 60)}h ${m % 60}m`;
};
const ids = (list) => (list.length ? list.join(", ") : "none");
function statusText({ record, proxy }, nowMs, file) {
const lines = [`triage key: ${record ? "issued" : "not issued"}`, `file: ${file}`];
const rotateDue = record && Date.parse(`${record.rotateBy}T00:00:00Z`) <= nowMs;
const retireDue = record ? record.retiring.filter((r) => Date.parse(r.retireAt) <= nowMs) : [];
if (record) {
lines.push(`key id: ${record.keyId}`, `issued: ${record.issuedAt} (age ${age(nowMs - Date.parse(record.issuedAt))})`, `rotate by: ${record.rotateBy}${rotateDue ? " (due)" : ""}`);
record.retiring.forEach((r, i) => {
const held = proxy.retiringHeld?.[i];
lines.push(`retiring: ${r.keyId}, live until a retire after ${r.retireAt}${Date.parse(r.retireAt) <= nowMs ? " (due)" : ""}${held === false ? " (not in the proxy list)" : ""}`);
});
}
const state = {
"not-set-up": "not set up for this instance (no proxy.json)",
"no-sentinel": "its key list has no sentinel for this instance",
unreadable: `not read (${proxy.code})`,
read: record ? (proxy.holdsKey ? "holds the key" : "does not hold the key") : "read",
}[proxy.state];
lines.push(`proxy: ${state}`);
if (proxy.fence) lines.push("proxy fence: set, an earlier proxy write has an uncertain outcome");
if (proxy.leftoverTemps) lines.push(`leftover temp files in the triage secrets directory: ${proxy.leftoverTemps} (the next issue, rotate or retire removes them)`);
if (proxy.orphanKeyIds?.length) lines.push(`keys in the Triage namespace the file does not hold: ${ids(proxy.orphanKeyIds)}`);
let next;
if (proxy.fence) next = "restart the proxy, then mosaic proxy fence clear --proxy-restarted";
else if (proxy.state !== "read") next = proxy.state === "not-set-up" ? "set up the model proxy first (mosaic proxy init)" : "check the proxy with mosaic proxy status";
else if (!record) next = "mosaic triage key issue";
else if (!proxy.holdsKey || rotateDue) next = "mosaic triage key rotate";
else if (retireDue.length || proxy.leftoverTemps || proxy.orphanKeyIds?.length || proxy.retiringHeld?.includes(false)) next = "mosaic triage key retire";
else next = "nothing";
lines.push(`next: ${next}`, "");
return lines.join("\n");
}
async function keyCommand(sub, args, io, deps) {
if (!KEY_VERBS.includes(sub)) throw usage("unknown triage key command");
if (args.length) throw usage(args[0].startsWith("--") ? "unknown option" : "unexpected argument");
try {
const system = deps.system ?? loadSystem({ env: io.env });
const dir = configDir(io.env);
const now = deps.now ? () => deps.now().getTime() : Date.now;
const opts = { configDir: dir, system, now, uid: deps.triageKeyUid, fileUid: deps.triageKeyFileUid, hook: deps.triageKeyHook };
if (sub === "status") {
io.stdout.write(statusText(await keyStatus(opts), now(), keyPaths(dir).file));
return;
}
if (sub === "issue") {
const r = await issueKey(opts);
io.stdout.write(`triage key issued: ${r.keyId}, rotate by ${r.rotateBy}\nfile: ${keyPaths(dir).file}\n`);
if (r.removedKeyIds.length) io.stdout.write(`removed keys this file did not hold: ${ids(r.removedKeyIds)}\n`);
return;
}
if (sub === "rotate") {
const r = await rotateKey(opts);
io.stdout.write(`triage key rotated: ${r.keyId}, rotate by ${r.rotateBy}\n`);
io.stdout.write(r.old.lost
? `old key ${r.old.keyId} was not in the proxy list; nothing to retire\n`
: `old key ${r.old.keyId} stays live until mosaic triage key retire runs at or after ${r.old.retireAt}\n`);
if (r.removedKeyIds.length) io.stdout.write(`removed: ${ids(r.removedKeyIds)}\n`);
return;
}
const r = await retireKeys(opts);
io.stdout.write(r.removedKeyIds.length ? `triage key retire: removed ${ids(r.removedKeyIds)}\n` : "triage key retire: nothing to remove\n");
} catch (e) { throw keyError(e); }
}
export function triageCommand(rest, io, deps = {}) {
refuseInsideAgent(process.env);
refuseInsideAgent(io.env);
const [noun, sub, ...args] = rest;
if (noun === "key" && sub !== undefined) return keyCommand(sub, args, io, deps);
if (noun !== "route" || !["show", "set"].includes(sub)) throw usage("unknown triage command");
const { values, positional } = parse(args);
if (positional.length) throw usage("unexpected argument");
+695
View File
@@ -0,0 +1,695 @@
// #1873 triage-key: `mosaic triage key issue|rotate|retire|status`, Triage's own model proxy key.
// A stand-in management server on loopback and a temporary config directory only; nothing here
// reads a real config, proxy or secret. Every stdout, stderr, receipt and file the verbs leave is
// scanned for every key the stand-in ever saw, at the end of each test and once more at the end.
import { test } from "node:test";
import assert from "node:assert/strict";
import { spawn, spawnSync } from "node:child_process";
import fs from "node:fs";
import http from "node:http";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createHash, randomBytes } from "node:crypto";
import { run } from "../src/cli.mjs";
import { AGENT_MARKERS } from "../src/transport.mjs";
import { DEFAULT_ROTATE_GRACE_MINUTES, keyPaths, retireAtFor, rotateByFor } from "../src/triage-key.mjs";
import { ModelKeys } from "../../bus/src/model-keys.mjs";
import { ManagementClient } from "../../model-proxy/src/client.mjs";
import { startBroker } from "../../bus/src/runtime.mjs";
import { busOrganization } from "../../bus/src/organization-context.mjs";
import { Registry } from "../../seat/src/session.mjs";
import { loadBusiness } from "../../organization/src/organization-config.mjs";
import { resolveInstance } from "../../organization/src/resolve.mjs";
import { businessDoc, rolesCopy, systemFor, writeJson } from "../../organization/tests/helpers.mjs";
import { humanEnv, io } from "./helpers.mjs";
const REPO = new URL("../../../", import.meta.url).pathname;
const NOW = Date.parse("2026-11-18T12:00:00.000Z");
const MIN = 60_000, HOUR = 60 * MIN, DAY = 24 * HOUR;
const MANAGE = "fx1873-manage";
const KEY_PATH = "/v8/management/config/access/api-keys";
const sha = (s) => createHash("sha256").update(s).digest("hex");
const tag = (dir) => sha(dir).slice(0, 12);
const id = (key) => sha(key).slice(0, 12);
const b64 = () => randomBytes(32).toString("base64url");
const iso = (ms) => new Date(ms).toISOString();
// Every output any verb produced in this file, and every Triage key the stand-in ever held.
const OUTPUTS = [];
const SEEN = new Set();
const VERBS = ["issue", "rotate", "retire", "status"];
function leaks(text, keys = SEEN) {
for (const key of keys) {
const secret = key.slice(-43);
if (text.includes(key) || text.includes(secret)) return true;
for (let i = 0; i + 8 <= secret.length; i++) if (text.includes(secret.slice(i, i + 8))) return true;
}
return false;
}
function walk(dir, skip, out = []) {
for (const name of fs.readdirSync(dir)) {
const p = join(dir, name), st = fs.lstatSync(p);
if (st.isDirectory()) walk(p, skip, out);
else if (st.isFile() && !skip.includes(p)) out.push(p);
}
return out;
}
async function fixture(t, { sentinel = true } = {}) {
const root = fs.realpathSync(fs.mkdtempSync(join(tmpdir(), "tk-")));
const configDir = join(root, "config");
fs.mkdirSync(join(configDir, "secrets"), { recursive: true, mode: 0o700 });
fs.chmodSync(configDir, 0o700); fs.chmodSync(join(configDir, "secrets"), 0o700);
fs.writeFileSync(join(configDir, "secrets", "proxy-management.key"), MANAGE, { mode: 0o600 });
fs.mkdirSync(join(root, "proxy"), { mode: 0o700 });
const prefix = `mst-${tag(configDir)}-triage-`;
const state = {
keys: [...(sentinel ? [`msk-${tag(configDir)}-sentinel-${b64()}`] : []), "foreign-client-key", `mso-${tag(configDir)}-demo-key-${b64()}`],
gets: 0, puts: [], ignorePut: false, dropSentinelOnPut: false, onGet: null,
};
const server = http.createServer(async (req, res) => {
if (req.url !== KEY_PATH) {
const key = req.headers.authorization?.slice(7);
res.writeHead(state.keys.includes(key) ? 200 : 401); res.end("{}"); return;
}
if (req.headers.authorization !== `Bearer ${MANAGE}`) { res.writeHead(401); res.end("{}"); return; }
if (req.method === "GET") { state.gets++; const body = JSON.stringify(state.keys); state.onGet?.(state.gets); res.end(body); return; }
const chunks = []; for await (const c of req) chunks.push(c);
const next = JSON.parse(Buffer.concat(chunks));
for (const k of next) if (/^mst-.*-[A-Za-z0-9_-]{43}$/.test(k)) SEEN.add(k);
state.puts.push(next);
if (state.dropSentinelOnPut) state.keys = next.filter((k) => !k.startsWith("msk-") || !k.includes("-sentinel-"));
else if (!state.ignorePut) state.keys = next;
res.end('{"status":"ok"}');
});
await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
const port = server.address().port;
const closers = [];
t.after(async () => {
for (const close of closers) await close();
server.closeAllConnections(); await new Promise((resolve) => server.close(resolve));
fs.rmSync(root, { recursive: true, force: true });
});
const f = {
root, configDir, prefix, state, server, port, closers, now: NOW,
system: { dataRoot: root, proxy: { port, binary: "/fixture-unused" } },
file: keyPaths(configDir).file, dir: keyPaths(configDir).dir,
env: (extra = {}) => humanEnv({ MOSAIC_CONFIG: join(configDir, "config.json"), ...extra }),
async cli(argv, { env, deps = {}, system = f.system } = {}) {
const s = io(env ?? f.env());
const code = await run(argv, s, { system, now: () => new Date(f.now), ...deps });
const out = { code, stdout: s.out.text, stderr: s.err.text };
OUTPUTS.push(out.stdout, out.stderr);
return out;
},
// The same verb in a child process that SIGKILLs itself at `stage`.
crash(argv, stage) {
const script = `
import { run } from ${JSON.stringify(new URL("../src/cli.mjs", import.meta.url).href)};
import { io } from ${JSON.stringify(new URL("./helpers.mjs", import.meta.url).href)};
const p = JSON.parse(process.env.TK_CRASH);
const s = io(process.env);
const code = await run(p.argv, s, { system: p.system, now: () => new Date(p.now),
triageKeyHook: (stage) => { if (stage === p.stage) process.kill(process.pid, "SIGKILL"); } });
process.stdout.write(s.out.text); process.stderr.write(s.err.text); process.exitCode = code;`;
const env = f.env({ TK_CRASH: JSON.stringify({ argv, stage, system: f.system, now: f.now }) });
return new Promise((resolve) => {
const child = spawn(process.execPath, ["--input-type=module", "-e", script], { env, stdio: ["ignore", "pipe", "pipe"] });
let stdout = "", stderr = "";
child.stdout.on("data", (d) => { stdout += d; });
child.stderr.on("data", (d) => { stderr += d; });
child.on("exit", (code, signal) => { OUTPUTS.push(stdout, stderr); resolve({ code, signal, stdout, stderr }); });
});
},
record: () => JSON.parse(fs.readFileSync(f.file, "utf8")),
triageKeys: () => state.keys.filter((k) => k.startsWith(prefix)),
receipts: () => {
const file = join(root, "proxy", "mosaic-keys.receipts.jsonl");
return fs.existsSync(file) ? fs.readFileSync(file, "utf8").split("\n").filter(Boolean).map((l) => JSON.parse(l)) : [];
},
fenced: () => fs.existsSync(join(root, "proxy", "mosaic-keys.uncertain")),
// No file under the fixture but the key file itself holds a key; neither does any output.
scan(alsoKeyFiles = []) {
for (const k of state.keys) if (/^mst-.*-[A-Za-z0-9_-]{43}$/.test(k)) SEEN.add(k);
for (const p of walk(root, [f.file, join(configDir, "secrets", "proxy-management.key"), ...alsoKeyFiles])) {
assert.equal(leaks(fs.readFileSync(p, "latin1")), false, `a key is in ${p}`);
}
for (const text of OUTPUTS) assert.equal(leaks(text), false, "a key is in an output");
},
};
return f;
}
const ok = (r) => assert.equal(r.code, 0, r.stderr);
const refusal = (r, code, words) => {
assert.equal(r.code, 3, r.stderr);
assert.equal(r.stdout, "");
assert.match(r.stderr, new RegExp(`^mosaic: ${code}: ${words ?? ""}`));
};
test("issue adds one key under the lock, writes the 0600 file, and records ids only", async (t) => {
const f = await fixture(t);
const before = [...f.state.keys];
const r = await f.cli(["triage", "key", "issue"]);
ok(r);
const rec = f.record();
assert.deepEqual(Object.keys(rec), ["version", "key", "keyId", "issuedAt", "rotateBy", "retiring"]);
assert.match(rec.key, new RegExp(`^${f.prefix}[A-Za-z0-9_-]{43}$`));
assert.equal(rec.keyId, id(rec.key));
assert.equal(rec.issuedAt, iso(NOW));
assert.equal(rec.rotateBy, "2026-12-18");
assert.deepEqual(rec.retiring, []);
assert.equal(r.stdout, `triage key issued: ${rec.keyId}, rotate by 2026-12-18\nfile: ${f.file}\n`);
assert.deepEqual(f.state.keys, [...before, rec.key]);
assert.equal(f.state.puts.length, 1);
const file = fs.lstatSync(f.file), dir = fs.lstatSync(f.dir);
assert.equal(file.mode & 0o777, 0o600); assert.equal(dir.mode & 0o777, 0o700);
assert.equal(file.uid, process.getuid()); assert.equal(dir.uid, process.getuid());
assert.deepEqual(f.receipts(), [{ op: "triage.key.issue", time: iso(NOW), addedKeyIds: [rec.keyId], removedKeyIds: [] }]);
assert.equal(f.fenced(), false);
assert.equal(fs.readFileSync(join(f.root, "proxy", "keys.generation"), "utf8"), "1\n");
// A second issue refuses and changes nothing; rotate is the verb for that.
const again = await f.cli(["triage", "key", "issue"]);
refusal(again, "triage-key-issued", "a Triage key is already issued; use mosaic triage key rotate\n$");
assert.equal(f.state.puts.length, 1);
assert.deepEqual(f.record(), rec);
f.scan();
});
test("status reads the file and the list and writes nothing", async (t) => {
const f = await fixture(t);
const none = await f.cli(["triage", "key", "status"]);
ok(none);
assert.equal(none.stdout, `triage key: not issued\nfile: ${f.file}\nproxy: read\nnext: mosaic triage key issue\n`);
ok(await f.cli(["triage", "key", "issue"]));
const rec = f.record(), puts = f.state.puts.length, receipts = f.receipts().length;
f.now = NOW + 3 * DAY + 4 * HOUR + 5 * MIN;
const shown = await f.cli(["triage", "key", "status"]);
ok(shown);
assert.equal(shown.stdout, [
"triage key: issued", `file: ${f.file}`, `key id: ${rec.keyId}`, `issued: ${iso(NOW)} (age 3d 4h 5m)`,
"rotate by: 2026-12-18", "proxy: holds the key", "next: nothing", "",
].join("\n"));
f.now = Date.parse("2026-12-18T00:00:00.000Z");
assert.match((await f.cli(["triage", "key", "status"])).stdout, /rotate by: 2026-12-18 \(due\)\n[^]*next: mosaic triage key rotate\n$/);
f.now = NOW + HOUR;
f.state.keys = f.state.keys.filter((k) => k !== rec.key);
assert.match((await f.cli(["triage", "key", "status"])).stdout, /proxy: does not hold the key\nnext: mosaic triage key rotate\n$/);
assert.equal(f.state.puts.length, puts);
assert.equal(f.receipts().length, receipts);
assert.equal(fs.readdirSync(join(f.root, "proxy")).includes("mosaic-keys.lock"), false);
f.scan();
});
test("rotate adds a new key and keeps the old one for the bus grace; retire removes it once due", async (t) => {
const f = await fixture(t);
refusal(await f.cli(["triage", "key", "rotate"]), "triage-key-not-issued", "no Triage key is issued; use mosaic triage key issue\n$");
assert.equal(f.state.puts.length, 0);
ok(await f.cli(["triage", "key", "issue"]));
const first = f.record();
f.now = NOW + HOUR;
const r = await f.cli(["triage", "key", "rotate"]);
ok(r);
const second = f.record();
const retireAt = iso(NOW + HOUR + 120 * MIN);
assert.notEqual(second.key, first.key);
assert.equal(second.issuedAt, iso(NOW + HOUR));
assert.deepEqual(second.retiring, [{ key: first.key, keyId: first.keyId, issuedAt: first.issuedAt, retireAt }]);
assert.equal(r.stdout, `triage key rotated: ${second.keyId}, rotate by 2026-12-18\nold key ${first.keyId} stays live until mosaic triage key retire runs at or after ${retireAt}\n`);
assert.deepEqual(f.triageKeys(), [first.key, second.key]);
f.now = NOW + HOUR + 119 * MIN;
const early = await f.cli(["triage", "key", "retire"]);
ok(early);
assert.equal(early.stdout, "triage key retire: nothing to remove\n");
assert.equal(f.state.puts.length, 2);
assert.match((await f.cli(["triage", "key", "status"])).stdout, new RegExp(`retiring: ${first.keyId}, live until a retire after ${retireAt}\\n[^]*next: nothing\\n$`));
f.now = NOW + HOUR + 120 * MIN;
assert.match((await f.cli(["triage", "key", "status"])).stdout, new RegExp(`retiring: ${first.keyId}, live until a retire after ${retireAt} \\(due\\)\\n[^]*next: mosaic triage key retire\\n$`));
const done = await f.cli(["triage", "key", "retire"]);
ok(done);
assert.equal(done.stdout, `triage key retire: removed ${first.keyId}\n`);
assert.deepEqual(f.triageKeys(), [second.key]);
assert.deepEqual(f.record(), { ...second, retiring: [] });
assert.deepEqual(f.receipts().map((e) => e.op), ["triage.key.issue", "triage.key.rotate", "triage.key.retire"]);
assert.deepEqual(f.receipts()[2].removedKeyIds, [first.keyId]);
f.scan();
});
test("rotate takes the grace from proxy.json, where the bus reads it, and drops an old key the proxy lost", async (t) => {
const f = await fixture(t);
f.system = { ...f.system, proxy: { ...f.system.proxy, rotateGraceMinutes: 30 } };
ok(await f.cli(["triage", "key", "issue"]));
const first = f.record();
ok(await f.cli(["triage", "key", "rotate"]));
const second = f.record();
assert.equal(second.retiring[0].retireAt, iso(NOW + 30 * MIN));
// The proxy lost the current key: rotate drops it at once, as the bus does for an agent key.
f.state.keys = f.state.keys.filter((k) => k !== second.key);
const r = await f.cli(["triage", "key", "rotate"]);
ok(r);
assert.match(r.stdout, new RegExp(`old key ${second.keyId} was not in the proxy list; nothing to retire\\n$`));
assert.deepEqual(f.record().retiring.map((e) => e.keyId), [first.keyId]);
// A retiring key past its time, or no longer in the list, leaves on the next rotate.
f.now = NOW + 31 * MIN;
ok(await f.cli(["triage", "key", "rotate"]));
const last = f.record();
assert.deepEqual(last.retiring.map((e) => e.keyId), [id(f.triageKeys()[0])]);
assert.equal(f.triageKeys().includes(first.key), false);
assert.equal(f.triageKeys().length, 2);
f.scan();
});
test("lifetime and grace match the bus's own ModelKeys for the same clock and proxy settings", async (t) => {
const runtime = fs.readFileSync(join(REPO, "packages/bus/src/runtime.mjs"), "utf8");
const literal = runtime.match(/rotateGraceMinutes: proxy\.rotateGraceMinutes \?\? (\d+)/);
assert.ok(literal, "the bus no longer reads rotateGraceMinutes as this test expects");
assert.equal(Number(literal[1]), DEFAULT_ROTATE_GRACE_MINUTES);
for (const grace of [undefined, 45]) {
const f = await fixture(t);
if (grace !== undefined) f.system = { ...f.system, proxy: { ...f.system.proxy, rotateGraceMinutes: grace } };
ok(await f.cli(["triage", "key", "issue"]));
f.now = NOW + 3 * HOUR;
ok(await f.cli(["triage", "key", "rotate"]));
const ours = f.record();
// The bus, driven the way runtime.mjs drives it, on its own in-memory list.
let keys = [`msk-${tag(f.configDir)}-sentinel-${b64()}`], record = null, now = NOW;
const credentials = { registerModel() {}, modelRecord: () => record, writeModel: (_, r) => { record = r; }, status: () => [] };
const client = { getKeys: async () => [...keys], putKeys: async (k) => { keys = [...k]; return { ok: true }; } };
const bus = new ModelKeys({ credentials, client, configDir: f.configDir, baseUrl: "http://127.0.0.1:1", now: () => now,
...(grace === undefined ? {} : { rotateGraceMinutes: f.system.proxy.rotateGraceMinutes ?? Number(literal[1]) }) });
await bus.reconcile();
await bus.ensure("demo/x");
assert.equal(record.rotateBy, rotateByFor(NOW));
now = NOW + 3 * HOUR;
await bus.rotate("demo/x");
assert.equal(ours.rotateBy, record.rotateBy);
assert.equal(ours.retiring[0].retireAt, record.retiring[0].retireAt);
assert.equal(ours.retiring[0].retireAt, retireAtFor(now, f.system.proxy));
f.scan();
}
});
test("an agent caller is refused before anything is read, for every key verb", async (t) => {
const f = await fixture(t);
for (const marker of AGENT_MARKERS) {
for (const verb of VERBS) {
const r = await f.cli(["triage", "key", verb], { env: { ...f.env(), [marker]: "1" } });
assert.equal(r.code, 3, marker);
assert.equal(r.stderr, `mosaic: refused: this runs only from a human shell, outside any agent run (found ${marker} in the environment)\n`);
}
const saved = process.env[marker];
process.env[marker] = "1";
try {
for (const verb of VERBS) {
const r = await f.cli(["triage", "key", verb]);
assert.equal(r.code, 3, marker);
assert.match(r.stderr, new RegExp(`found ${marker} in the environment`));
}
} finally { if (saved === undefined) delete process.env[marker]; else process.env[marker] = saved; }
}
assert.equal(f.state.gets, 0);
assert.equal(f.state.puts.length, 0);
assert.equal(fs.existsSync(f.dir), false);
assert.deepEqual(fs.readdirSync(join(f.root, "proxy")), []);
});
test("usage: unknown key verbs, options and arguments exit 4 and echo no value", async (t) => {
const f = await fixture(t);
for (const argv of [["triage", "key"], ["triage", "key", "nope"], ["triage", "key", "issue", "extra"], ["triage", "key", "status", "--json"]]) {
const r = await f.cli(argv);
assert.equal(r.code, 4, argv.join(" "));
assert.equal(r.stdout, "");
assert.match(r.stderr, /\nusage: mosaic triage route show [^\n]*\| key issue \| rotate \| retire \| status\n$/);
}
const r = await f.cli(["triage", "key", "rotate", "--key", "secret-value"]);
assert.equal(r.code, 4);
assert.match(r.stderr, /^mosaic: unknown option\n/);
assert.doesNotMatch(r.stderr, /secret-value/);
assert.equal(f.state.gets, 0);
});
test("an unsafe key store refuses every verb before any proxy write", async (t) => {
const uid = process.getuid();
const cases = [
["file mode", (f) => fs.chmodSync(f.file, 0o644), {}, "the key file mode is not 0600"],
["file owner", () => {}, { triageKeyFileUid: uid + 1 }, "the key file is not owned by this user"],
["file link", (f) => { fs.renameSync(f.file, join(f.dir, "real")); fs.symlinkSync(join(f.dir, "real"), f.file); }, {}, "the key file is a link"],
["file is a directory", (f) => { fs.rmSync(f.file); fs.mkdirSync(f.file, { mode: 0o700 }); }, {}, "the key file is not a regular file"],
["file is a FIFO", (f) => { fs.rmSync(f.file); assert.equal(spawnSync("mkfifo", ["-m", "600", f.file]).status, 0); }, {}, "the key file is not a regular file"],
["hard link", (f) => fs.linkSync(f.file, join(f.dir, "second")), {}, "the key file has more than one link"],
["not JSON", (f) => fs.writeFileSync(f.file, "{"), {}, "the key file is not JSON"],
["extra field", (f) => fs.writeFileSync(f.file, JSON.stringify({ ...f.record(), extra: 1 })), {}, "the key file does not match the key shape"],
["another instance's key", (f) => { const r = f.record(); const key = `mst-${tag("/elsewhere")}-triage-${b64()}`; fs.writeFileSync(f.file, JSON.stringify({ ...r, key, keyId: id(key) })); }, {}, "the key file does not match the key shape"],
["wrong key id", (f) => fs.writeFileSync(f.file, JSON.stringify({ ...f.record(), keyId: "000000000000" })), {}, "the key file does not match the key shape"],
["too large", (f) => fs.writeFileSync(f.file, " ".repeat(64 * 1024 + 1)), {}, "the key file is too large"],
["triage directory mode", (f) => fs.chmodSync(f.dir, 0o755), {}, "the triage secrets directory mode is not 0700"],
["triage directory link", (f) => { fs.renameSync(f.dir, join(f.configDir, "secrets", "moved")); fs.symlinkSync(join(f.configDir, "secrets", "moved"), f.dir); }, {}, "the triage secrets directory is a link"],
["triage directory is a file", (f) => { fs.rmSync(f.dir, { recursive: true }); fs.writeFileSync(f.dir, "", { mode: 0o600 }); }, {}, "the triage secrets directory is not a directory"],
["secrets mode", (f) => fs.chmodSync(join(f.configDir, "secrets"), 0o750), {}, "the secrets directory mode is not 0700"],
["owner", () => {}, { triageKeyUid: uid + 1 }, "the secrets directory is not owned by this user"],
];
for (const [name, spoil, deps, words] of cases) {
const f = await fixture(t);
ok(await f.cli(["triage", "key", "issue"]));
spoil(f);
const keys = [...f.state.keys], puts = f.state.puts.length;
for (const verb of VERBS) {
const r = await f.cli(["triage", "key", verb], { deps });
refusal(r, "triage-key-unreadable", `the Triage key store is unreadable: ${words.replace(/[.()]/g, "\\$&")}\n$`);
assert.equal(f.state.puts.length, puts, `${name}: ${verb}`);
}
assert.deepEqual(f.state.keys, keys);
// The spoils above move or link the key file itself; those copies are the key file.
f.scan([join(f.dir, "real"), join(f.dir, "second"), join(f.configDir, "secrets", "moved", "model")]);
}
// No secrets directory at all: status says not issued; issue refuses before the proxy changes.
const f = await fixture(t);
fs.rmSync(join(f.configDir, "secrets"), { recursive: true });
fs.mkdirSync(join(f.configDir, "secrets"), { mode: 0o700 });
fs.writeFileSync(join(f.configDir, "secrets", "proxy-management.key"), MANAGE, { mode: 0o600 });
fs.chmodSync(join(f.configDir, "secrets"), 0o755);
refusal(await f.cli(["triage", "key", "issue"]), "triage-key-unreadable", "the Triage key store is unreadable: the secrets directory mode is not 0700\n$");
assert.equal(f.state.puts.length, 0);
});
test("the instance proxy lock, the fence, the sentinel and the proxy itself each refuse plainly", async (t) => {
// Lock held by a live process: busy after the lock's wait, and nothing changes.
const f = await fixture(t);
const lockFile = join(f.root, "proxy", "mosaic-keys.lock");
ok(await f.cli(["triage", "key", "status"]));
fs.writeFileSync(lockFile, `${JSON.stringify({ pid: process.pid, token: "a".repeat(32) })}\n`, { mode: 0o600 });
const busy = await f.cli(["triage", "key", "issue"]);
refusal(busy, "proxy-lock-busy", "another proxy writer holds the instance proxy lock; try again when it ends, and if no writer is running, see mosaic proxy fence status\n$");
assert.equal(f.state.gets, 1);
// A lock left by a process that is gone is reaped, and the verb goes ahead.
const dead = spawnSync(process.execPath, ["-e", ""]).pid;
fs.writeFileSync(lockFile, `${JSON.stringify({ pid: dead, token: "b".repeat(32) })}\n`, { mode: 0o600 });
ok(await f.cli(["triage", "key", "issue"]));
assert.equal(fs.existsSync(lockFile), false);
assert.equal(f.triageKeys().length, 1);
fs.rmSync(f.file);
f.state.keys = f.state.keys.filter((k) => !k.startsWith(f.prefix));
// A set fence: every write refuses, status names it and the way out.
const putsBefore = f.state.puts.length;
fs.writeFileSync(join(f.root, "proxy", "mosaic-keys.uncertain"), "{}\n", { mode: 0o600 });
for (const verb of ["issue", "rotate", "retire"]) refusal(await f.cli(["triage", "key", verb]), "proxy-uncertain");
assert.match((await f.cli(["triage", "key", "status"])).stdout, /proxy fence: set, an earlier proxy write has an uncertain outcome\nnext: restart the proxy, then mosaic proxy fence clear --proxy-restarted\n$/);
assert.equal(f.state.puts.length, putsBefore);
// No sentinel for this instance (another instance's does not count).
const g = await fixture(t, { sentinel: false });
g.state.keys.push(`msk-${tag("/elsewhere")}-sentinel-${b64()}`);
refusal(await g.cli(["triage", "key", "issue"]), "triage-key-proxy-open");
assert.match((await g.cli(["triage", "key", "status"])).stdout, /proxy: its key list has no sentinel for this instance\nnext: check the proxy with mosaic proxy status\n$/);
assert.equal(g.state.puts.length, 0);
// No proxy set up, no management key, proxy down.
const h = await fixture(t);
const bare = { dataRoot: h.root };
refusal(await h.cli(["triage", "key", "issue"], { system: bare }), "triage-key-no-proxy");
assert.equal((await h.cli(["triage", "key", "status"], { system: bare })).stdout,
`triage key: not issued\nfile: ${h.file}\nproxy: not set up for this instance (no proxy.json)\nnext: set up the model proxy first (mosaic proxy init)\n`);
fs.chmodSync(join(h.configDir, "secrets", "proxy-management.key"), 0o644);
refusal(await h.cli(["triage", "key", "issue"]), "proxy-management-key-invalid");
fs.chmodSync(join(h.configDir, "secrets", "proxy-management.key"), 0o600);
h.server.closeAllConnections(); await new Promise((resolve) => h.server.close(resolve));
h.server.close = (cb) => cb();
refusal(await h.cli(["triage", "key", "issue"]), "proxy-client-down");
assert.match((await h.cli(["triage", "key", "status"])).stdout, /proxy: not read \(proxy-client-down\)\n/);
assert.equal(fs.existsSync(h.file), false);
f.scan(); g.scan(); h.scan();
});
test("a write the proxy does not confirm keeps the fence; a list that moves mid-verb is not overwritten", async (t) => {
const f = await fixture(t);
f.state.ignorePut = true;
refusal(await f.cli(["triage", "key", "issue"]), "triage-key-unconfirmed");
assert.equal(f.fenced(), true);
assert.equal(fs.existsSync(f.file), false);
// The store's directory was made and checked before the proxy changed, not after.
assert.equal(fs.lstatSync(f.dir).mode & 0o777, 0o700);
assert.deepEqual(f.receipts(), []);
// A PUT the proxy takes but whose read-back has lost this instance's sentinel is not confirmed.
const s = await fixture(t);
s.state.dropSentinelOnPut = true;
refusal(await s.cli(["triage", "key", "issue"]), "triage-key-unconfirmed");
assert.equal(s.fenced(), true);
assert.equal(fs.existsSync(s.file), false);
s.scan();
const g = await fixture(t);
g.state.onGet = (n) => { if (n === 1) g.state.keys = [...g.state.keys, "late-foreign-key"]; };
refusal(await g.cli(["triage", "key", "issue"]), "triage-key-raced");
assert.equal(g.state.puts.length, 0);
assert.equal(g.fenced(), false);
f.scan(); g.scan();
});
test("keys outside the Triage namespace are never touched; Triage keys the file does not hold are removed", async (t) => {
const f = await fixture(t);
const agent = `msk-${tag(f.configDir)}-${b64()}`, other = `mst-${tag("/elsewhere")}-triage-${b64()}`;
const orphan = `${f.prefix}${b64()}`, odd = `${f.prefix}short`;
f.state.keys.push(agent, other, orphan, odd);
const foreign = f.state.keys.filter((k) => k !== orphan && k !== odd);
const r = await f.cli(["triage", "key", "issue"]);
ok(r);
// A malformed name under the Triage prefix is still Triage's, as the bus treats its own prefix.
assert.match(r.stdout, new RegExp(`removed keys this file did not hold: ${id(orphan)}, ${id(odd)}\\n$`));
for (const verb of ["rotate", "retire"]) { f.now += 3 * HOUR; ok(await f.cli(["triage", "key", verb])); }
for (const k of foreign) assert.ok(f.state.keys.includes(k), "a key outside the namespace left the list");
assert.equal(f.state.keys.includes(orphan), false);
assert.equal(f.state.keys.includes(odd), false);
assert.deepEqual(f.triageKeys(), [f.record().key]);
f.scan();
});
test("crash after the PUT, before the read-back: the fence holds; status says so; clear, then issue recovers", async (t) => {
const f = await fixture(t);
const crashed = await f.crash(["triage", "key", "issue"], "put");
assert.equal(crashed.signal, "SIGKILL");
assert.equal(f.fenced(), true);
assert.equal(f.triageKeys().length, 1);
assert.equal(fs.existsSync(f.file), false);
const status = await f.cli(["triage", "key", "status"]);
ok(status);
assert.match(status.stdout, new RegExp(`proxy fence: set[^\\n]*\\nkeys in the Triage namespace the file does not hold: ${id(f.triageKeys()[0])}\\nnext: restart the proxy, then mosaic proxy fence clear --proxy-restarted\\n$`));
refusal(await f.cli(["triage", "key", "issue"]), "proxy-uncertain");
ok(await f.cli(["proxy", "fence", "clear", "--proxy-restarted"]));
assert.equal(f.fenced(), false);
const stray = f.triageKeys()[0];
const r = await f.cli(["triage", "key", "issue"]);
ok(r);
assert.match(r.stdout, new RegExp(`removed keys this file did not hold: ${id(stray)}\\n$`));
assert.deepEqual(f.triageKeys(), [f.record().key]);
f.scan();
});
test("crash after the read-back, before the file write: no fence; status names the stray key; issue removes it", async (t) => {
const f = await fixture(t);
assert.equal((await f.crash(["triage", "key", "issue"], "write")).signal, "SIGKILL");
assert.equal(f.fenced(), false);
assert.equal(fs.existsSync(f.file), false);
const [stray] = f.triageKeys();
assert.match((await f.cli(["triage", "key", "status"])).stdout,
new RegExp(`^triage key: not issued\\n[^]*keys in the Triage namespace the file does not hold: ${id(stray)}\\nnext: mosaic triage key issue\\n$`));
ok(await f.cli(["triage", "key", "issue"]));
assert.deepEqual(f.triageKeys(), [f.record().key]);
// The same crash in rotate: the file keeps the old key, the stray new one is named, retire removes it.
const held = f.record();
assert.equal((await f.crash(["triage", "key", "rotate"], "write")).signal, "SIGKILL");
assert.deepEqual(f.record(), held);
const strayNew = f.triageKeys().find((k) => k !== held.key);
assert.match((await f.cli(["triage", "key", "status"])).stdout,
new RegExp(`proxy: holds the key\\nkeys in the Triage namespace the file does not hold: ${id(strayNew)}\\nnext: mosaic triage key retire\\n$`));
const retired = await f.cli(["triage", "key", "retire"]);
ok(retired);
assert.equal(retired.stdout, `triage key retire: removed ${id(strayNew)}\n`);
assert.deepEqual(f.triageKeys(), [held.key]);
f.scan();
});
test("file written but the old key not yet retired: status shows it due; a crash inside retire recovers on the next retire", async (t) => {
const f = await fixture(t);
ok(await f.cli(["triage", "key", "issue"]));
const first = f.record();
ok(await f.cli(["triage", "key", "rotate"]));
f.now = NOW + 121 * MIN;
assert.match((await f.cli(["triage", "key", "status"])).stdout, new RegExp(`retiring: ${first.keyId}, live until a retire after [^\\n]* \\(due\\)\\n[^]*next: mosaic triage key retire\\n$`));
// retire dies after the proxy dropped the key but before the file was rewritten.
assert.equal((await f.crash(["triage", "key", "retire"], "write")).signal, "SIGKILL");
assert.equal(f.state.keys.includes(first.key), false);
assert.equal(f.record().retiring.length, 1);
assert.match((await f.cli(["triage", "key", "status"])).stdout, /\(due\) \(not in the proxy list\)\n[^]*next: mosaic triage key retire\n$/);
const puts = f.state.puts.length;
const r = await f.cli(["triage", "key", "retire"]);
ok(r);
assert.equal(f.state.puts.length, puts);
assert.deepEqual(f.record().retiring, []);
// And a retire killed after its PUT leaves the fence, like any other write.
ok(await f.cli(["triage", "key", "rotate"]));
f.now += 121 * MIN;
assert.equal((await f.crash(["triage", "key", "retire"], "put")).signal, "SIGKILL");
assert.equal(f.fenced(), true);
refusal(await f.cli(["triage", "key", "retire"]), "proxy-uncertain");
ok(await f.cli(["proxy", "fence", "clear", "--proxy-restarted"]));
ok(await f.cli(["triage", "key", "retire"]));
assert.deepEqual(f.record().retiring, []);
assert.deepEqual(f.triageKeys(), [f.record().key]);
f.scan();
});
test("a file that cannot be written after the proxy changed is reported with its recovery", async (t) => {
const f = await fixture(t);
ok(await f.cli(["triage", "key", "status"]));
fs.mkdirSync(f.dir, { mode: 0o700 });
const r = await f.cli(["triage", "key", "issue"], { deps: { triageKeyHook: (stage) => { if (stage === "write") fs.chmodSync(f.dir, 0o755); } } });
refusal(r, "triage-key-unwritten", "the proxy list changed but ");
assert.match(r.stderr, /run mosaic triage key issue again, which clears what this run left\n$/);
assert.equal(f.triageKeys().length, 1);
fs.chmodSync(f.dir, 0o700);
ok(await f.cli(["triage", "key", "issue"]));
assert.deepEqual(f.triageKeys(), [f.record().key]);
f.scan();
});
test("crash after the temp file, before its rename: status counts it; the next verb removes it", async (t) => {
const f = await fixture(t);
assert.equal((await f.crash(["triage", "key", "issue"], "rename")).signal, "SIGKILL");
assert.equal(fs.existsSync(f.file), false);
const temps = fs.readdirSync(f.dir).filter((n) => n.endsWith(".tmp"));
assert.equal(temps.length, 1);
assert.match(temps[0], /^\.model\.\d+\.[0-9a-f]{12}\.tmp$/);
// The temp file holds the key the proxy now has; that is why it must go.
const [stray] = f.triageKeys();
assert.ok(fs.readFileSync(join(f.dir, temps[0]), "utf8").includes(stray));
const status = await f.cli(["triage", "key", "status"]);
ok(status);
assert.match(status.stdout, new RegExp(`leftover temp files in the triage secrets directory: 1 \\(the next issue, rotate or retire removes them\\)\\nkeys in the Triage namespace the file does not hold: ${id(stray)}\\nnext: mosaic triage key issue\\n$`));
// Something else in the directory is not a leftover and stays.
fs.writeFileSync(join(f.dir, "notes.tmp"), "", { mode: 0o600 });
ok(await f.cli(["triage", "key", "issue"]));
assert.deepEqual(fs.readdirSync(f.dir).sort(), ["model", "notes.tmp"]);
assert.deepEqual(f.triageKeys(), [f.record().key]);
// Retire, with nothing else to do, still removes a leftover.
assert.equal((await f.crash(["triage", "key", "rotate"], "rename")).signal, "SIGKILL");
ok(await f.cli(["triage", "key", "retire"]));
assert.deepEqual(fs.readdirSync(f.dir).sort(), ["model", "notes.tmp"]);
assert.doesNotMatch((await f.cli(["triage", "key", "status"])).stdout, /leftover/);
f.scan();
});
test("the receipts file is checked before the proxy changes; a receipt that fails after is named", async (t) => {
const receipts = (f) => join(f.root, "proxy", "mosaic-keys.receipts.jsonl");
const spoils = [
["mode", (f) => { fs.writeFileSync(receipts(f), "", { mode: 0o644 }); fs.chmodSync(receipts(f), 0o644); }, "the proxy receipts file mode is not 0600"],
["link", (f) => { fs.writeFileSync(join(f.root, "elsewhere"), "", { mode: 0o600 }); fs.symlinkSync(join(f.root, "elsewhere"), receipts(f)); }, "the proxy receipts file cannot be opened"],
["FIFO", (f) => assert.equal(spawnSync("mkfifo", ["-m", "600", receipts(f)]).status, 0), "the proxy receipts file cannot be opened"],
];
for (const [name, spoil, words] of spoils) {
const f = await fixture(t);
spoil(f);
refusal(await f.cli(["triage", "key", "issue"]), "triage-key-receipts-unsafe", `${words}; nothing was changed\n$`);
assert.equal(f.state.puts.length, 0, name);
assert.equal(f.fenced(), false, name);
assert.equal(fs.existsSync(f.file), false, name);
f.scan();
}
// Safe before the change, unsafe at the receipt: the change stands and the refusal says so.
const f = await fixture(t);
const r = await f.cli(["triage", "key", "issue"], { deps: { triageKeyHook: (stage) => { if (stage === "receipt") fs.chmodSync(receipts(f), 0o644); } } });
refusal(r, "triage-key-unrecorded", "the change was made but its receipt line was not written; run mosaic triage key status\n$");
assert.deepEqual(f.triageKeys(), [f.record().key]);
assert.equal(f.fenced(), false);
assert.equal(fs.readFileSync(receipts(f), "utf8"), "");
f.scan();
});
test("a directory sync that fails after the rename is named, and the receipt is still written", async (t) => {
const f = await fixture(t);
const r = await f.cli(["triage", "key", "issue"], { deps: { triageKeyHook: (stage) => { if (stage === "sync") throw new Error("EIO"); } } });
refusal(r, "triage-key-unsynced", `${f.file.replace(/[.]/g, "\\.")} was written but its directory could not be synced, so it may not survive a power loss; run mosaic triage key status\n$`);
const rec = f.record();
assert.deepEqual(f.triageKeys(), [rec.key]);
assert.deepEqual(f.receipts().map((e) => e.addedKeyIds), [[rec.keyId]]);
assert.equal(f.fenced(), false);
assert.match((await f.cli(["triage", "key", "status"])).stdout, /proxy: holds the key\nnext: nothing\n$/);
f.scan();
});
test("an unexpected error prints fixed words, never the error's own text", async (t) => {
const f = await fixture(t);
const secret = `mst-${tag(f.configDir)}-triage-${b64()}`;
SEEN.add(secret);
// Outside the proxy write: the verb's own fallback line.
const broken = { dataRoot: f.root, get proxy() { throw new Error(`boom ${secret}`); } };
for (const verb of VERBS) {
const r = await f.cli(["triage", "key", verb], { system: broken });
assert.equal(r.code, 3, verb);
assert.equal(r.stderr, "mosaic: triage-key-failed: the key verb stopped on an unexpected error; run mosaic triage key status\n");
}
// Inside the proxy write: the write's own fixed line, and the fence stays for a human.
const r = await f.cli(["triage", "key", "issue"], { deps: { triageKeyHook: (stage) => { if (stage === "put") throw new Error(`boom ${secret}`); } } });
refusal(r, "triage-key-failed", "the proxy write failed\n$");
assert.equal(f.fenced(), true);
f.scan();
});
// The org bus reconciles its own `mso-` namespace at start and a no-org bus its `msk-` one; a
// Triage key sits in neither, so both leave it. Stray keys in each namespace show the reconcile ran.
test("an org bus that starts and reconciles, and a no-org reconcile, both leave the Triage key alone", async (t) => {
const f = await fixture(t);
ok(await f.cli(["triage", "key", "issue"]));
const triage = f.record().key;
const strayOrg = `mso-${tag(f.configDir)}-demo-key-${b64()}`;
f.state.keys.push(strayOrg);
const rolesDir = rolesCopy(f.root), system = systemFor(f.root), doc = businessDoc(f.root, "demo");
doc.plugins = { ...doc.plugins, harness: ["pi"] };
const file = writeJson(join(f.configDir, "businesses", "demo.json"), doc);
assert.ok(file);
const business = loadBusiness("demo", { dir: f.configDir, rolesDir }), resolved = {};
for (const instance of Object.keys(business.roles)) resolved[instance] = resolveInstance({ system, business, instance });
fs.mkdirSync(join(f.root, "data"), { mode: 0o700 });
new Registry(join(f.root, "data"));
const bus = await startBroker({ dataRoot: join(f.root, "data"), businesses: { demo: busOrganization(business, resolved) },
organization: { dir: f.configDir, rolesDir, system }, proxy: { port: f.port, binary: "/fixture-unused" },
org: { id: "demo" }, instanceDataRoot: f.root, instanceConfigDir: f.configDir });
f.closers.push(() => bus.close());
const orgKey = (await bus.modelPreflight({ business: "demo", role: "chief" })).key;
assert.ok(orgKey.startsWith(`mso-${tag(f.configDir)}-demo-key-`));
assert.equal(f.state.keys.includes(strayOrg), false, "the org reconcile did not run");
assert.ok(f.state.keys.includes(triage), "the org bus removed the Triage key");
// A no-org reconcile on the same instance.
const strayAgent = `msk-${tag(f.configDir)}-${b64()}`;
f.state.keys.push(strayAgent);
const client = new ManagementClient({ baseUrl: `http://127.0.0.1:${f.port}`, key: MANAGE });
const keys = new ModelKeys({ credentials: { status: () => [], modelRecord: () => null }, client, configDir: f.configDir, baseUrl: `http://127.0.0.1:${f.port}` });
await keys.reconcile();
assert.equal(f.state.keys.includes(strayAgent), false, "the no-org reconcile did not run");
assert.ok(f.state.keys.includes(triage), "the no-org reconcile removed the Triage key");
// Triage still works under the shared lock while the org bus runs, and leaves the org key.
f.now += HOUR;
ok(await f.cli(["triage", "key", "rotate"]));
assert.ok(f.state.keys.includes(orgKey));
assert.match((await f.cli(["triage", "key", "status"])).stdout, /proxy: holds the key\n/);
f.scan();
});
test("scripts/mosaic routes triage key status with the instance's own config directory", (t) => {
const home = fs.mkdtempSync(join(tmpdir(), "tkm-"));
t.after(() => fs.rmSync(home, { recursive: true, force: true }));
const env = humanEnv({ HOME: home, MOSAIC_CONFIG: join(home, ".config/mosaic-tktest/config.json") });
const setup = spawnSync(join(REPO, "scripts/bootstrap.sh"), ["--instance", "tktest"], { env, encoding: "utf8" });
assert.equal(setup.status, 0, setup.stderr);
const r = spawnSync(join(REPO, "scripts/mosaic"), ["triage", "key", "status"], { env, encoding: "utf8", timeout: 60000 });
OUTPUTS.push(r.stdout, r.stderr);
assert.equal(r.status, 0, r.stderr);
assert.equal(r.stdout, `triage key: not issued\nfile: ${join(home, ".config/mosaic-tktest/secrets/triage/model")}\nproxy: not set up for this instance (no proxy.json)\nnext: set up the model proxy first (mosaic proxy init)\n`);
const refused = spawnSync(join(REPO, "scripts/mosaic"), ["triage", "key", "issue"], { env: { ...env, CLAUDECODE: "1" }, encoding: "utf8", timeout: 60000 });
assert.equal(refused.status, 3);
assert.match(refused.stderr, /found CLAUDECODE in the environment/);
});
test("no output of any verb in this file holds any key the stand-in saw", () => {
assert.ok(SEEN.size >= 20, `only ${SEEN.size} keys were seen`);
assert.ok(OUTPUTS.length >= 100);
for (const text of OUTPUTS) assert.equal(leaks(text), false);
});
+18 -1
View File
@@ -38,6 +38,7 @@ import { createTriageHost } from './triage-host.mjs';
import { composeSnapshot, readSystemConfig, readOrgConfig } from './triage-snapshot.mjs';
import { busesStopped } from '../../cli/src/proxy-admin-host.mjs';
import { agentLevelsWriteRoutes } from './agent-levels-write.mjs';
import { triageRouteRoutes } from './triage-route-http.mjs';
import { RESTART_EXIT } from '../../cli/src/org-console.mjs';
import { ID_PATTERN, systemOnly } from '../../organization/src/vocabulary.mjs';
import { selectorScope } from '../../organization/src/selector.mjs';
@@ -262,8 +263,24 @@ try {
const trail = system ? actionTrail(join(system.dataRoot, 'console')) : null;
// #1873 A2: Settings > Agents writes for the root and project levels, beside the levels read.
const agentLevelRoutes = organizations ? [agentLevelsWriteRoutes({ configDir: configDir(), rolesDir: rolesDir(), trail })] : [];
// #1873 triage-settings: Settings > Triage reads and saves the route store for the instance owner.
// The model is checked against the catalogue each active organization's bus answers with, as an
// organization's route change is. The Triage host above still gets route: null; saving starts nothing.
const triageRoutes = system && (devOwner || ssoOn) ? [triageRouteRoutes({
dataRoot: system.dataRoot, log: line => console.log(line),
catalogues: async () => {
const read = async call => { try { return (await call('routes'))?.catalogue ?? { status: 'unknown' }; } catch { return { status: 'bus-unavailable' }; } };
if (!organizations) return ownerCall ? [await read(ownerCall)] : [];
// Every active organization is asked at once, so a hung bus costs one timeout, not one each.
const active = (await listOrgs({ configDir: configDir(), dataRoot: system.dataRoot, instance })).filter(org => org.state === 'active');
return Promise.all(active.map(org => {
let paths; try { paths = orgPaths(org.root); } catch { return { status: 'bus-unavailable' }; }
return read(busCaller({ dataRoot: org.root, socket: paths.socket, business: org.id, timeoutMs: 5000 }));
}));
},
})] : [];
try {
server = await startServer({ restart, trail, ...(lan ? { host: lan.bind, devAllowHosts: lan.allowHosts } : {}), board: options.board, port: options.port, bus, reads, preview: options.preview, sso, proof, send, settingsWrite, projectFiles, projectFile, libraryFile, libraryStatus, librarySync, libraryRevert, librarySave, projectSave, chat, nameWrite, assistantStop, instancesWrite, routesWrite, agentWrite, proxyCall, proxyAdmin, triage, processing, decisionWrite, requestRead, requestWrite, userOptions, federation: system?.federation ?? null, organizations, agentLevels, routes: agentLevelRoutes, preselect: options.business ?? null });
server = await startServer({ restart, trail, ...(lan ? { host: lan.bind, devAllowHosts: lan.allowHosts } : {}), board: options.board, port: options.port, bus, reads, preview: options.preview, sso, proof, send, settingsWrite, projectFiles, projectFile, libraryFile, libraryStatus, librarySync, libraryRevert, librarySave, projectSave, chat, nameWrite, assistantStop, instancesWrite, routesWrite, agentWrite, proxyCall, proxyAdmin, triage, processing, decisionWrite, requestRead, requestWrite, userOptions, federation: system?.federation ?? null, organizations, agentLevels, routes: [...agentLevelRoutes, ...triageRoutes], preselect: options.business ?? null });
} catch (err) { sso?.close(); throw err; }
if (testerSocket) {
const call = busCaller({ dataRoot: system.dataRoot, socket: socketPath(system.dataRoot), business: options.business ?? null, cli: TESTER_CLI });
+11 -6
View File
@@ -321,7 +321,7 @@
// render. That read marks nothing. Row 150 (#1644): since row 142, `mosaic talk` prints only the
// reply to its own message, so a reply to a Console send stays unread in `mosaic inbox` until
// the human marks it read with `mosaic inbox --read`.
let isTester = false, orgSetup = false;
let isTester = false, orgSetup = false, triageOwner = false;
// Both Docs and project direction use the same owner-only records setup action.
window.MosaicRecords = {
markup(project) {
@@ -1431,13 +1431,13 @@
});
});
// Organizations is writable when the server allows org setup, which it does in organization mode where send is off.
const readonlyPage = page => !['appearance','business','console'].includes(page) && !(page === 'organizations' && orgSetup) && !canSend && !isTester;
const settingsPages = [['appearance','Appearance'],['business',window.MosaicOrg ? 'Organization' : 'Business'],['limits','Launch limits'],['organizations','Organizations'],['routing','Routing'],['agents','Agents'],['assistant','Assistant'],['tokens','Token use'],['model-proxy','Model proxy'],['console','Console'],['about','About']];
const readonlyPage = page => !['appearance','business','console','triage'].includes(page) && !(page === 'organizations' && orgSetup) && !canSend && !isTester;
const settingsPages = [['appearance','Appearance'],['business',window.MosaicOrg ? 'Organization' : 'Business'],['limits','Launch limits'],['organizations','Organizations'],['routing','Routing'],['agents','Agents'],['assistant','Assistant'],['tokens','Token use'],['model-proxy','Model proxy'],['triage','Triage'],['console','Console'],['about','About']];
function markSettingsDraft(page,dirty){const link=document.querySelector(`#settings-nav a[href="#/settings/${page}"]`), dot='<span class="settings-dot"><span class="sr-only">unsaved changes</span></span>';if(link && dirty && !link.querySelector('.settings-dot'))link.insertAdjacentHTML('beforeend',dot);const button=$('settings-disclosure');if(dirty && link?.getAttribute('aria-current')==='page' && button && !button.querySelector('.settings-dot'))button.querySelector('strong').insertAdjacentHTML('beforeend',dot);}
function settingsShell(page, body, helper='') {
const pages=isTester ? [['appearance','Appearance'],['business','Business'],['routing','Routing'],['agents','Agents'],['names','Seat names']] : settingsPages;
const pages=isTester ? [['appearance','Appearance'],['business','Business'],['routing','Routing'],['agents','Agents'],['names','Seat names']] : settingsPages.filter(([id])=>id!=='triage'||triageOwner);
const label=pages.find(([id])=>id===page)?.[1] || 'Appearance';
const isDirty=id=>(id==='routing'&&routesDraft)||(id==='limits'&&settingsInputs)||(id==='assistant'&&(Object.keys(assistantDrafts).length||Object.keys(nameDrafts).length))||(id==='agents'&&(instanceDrafts.add||instanceDrafts.change||window.MosaicAgentLevels?.hasDraft()));
const isDirty=id=>(id==='routing'&&routesDraft)||(id==='limits'&&settingsInputs)||(id==='assistant'&&(Object.keys(assistantDrafts).length||Object.keys(nameDrafts).length))||(id==='agents'&&(instanceDrafts.add||instanceDrafts.change||window.MosaicAgentLevels?.hasDraft()))||(id==='triage'&&window.MosaicTriageRoute?.hasDraft());
const dot='<span class="settings-dot"><span class="sr-only">unsaved changes</span></span>';
const links=pages.map(([id,name],i)=>`${i===0?'<div class="settings-group">Your browser</div>':i===1?'<div class="settings-group">This instance</div>':''}<a href="#/settings/${id}"${id===page?' aria-current="page"':''}>${name}${isDirty(id)?dot:''}</a>`).join('') + (!isTester ? '<div class="settings-group">Members and invites <span class="muted">not built</span></div>' : '');
return `${head('Settings','Your browser and this instance.')}<div class="settings-layout"><div><button type="button" id="settings-disclosure" aria-expanded="false" aria-controls="settings-nav"><small>Section</small><strong>${label}${isDirty(page)?dot:''}</strong><span aria-hidden="true">⌄</span></button><nav id="settings-nav" aria-label="Settings sections">${links}</nav></div><div class="settings-content settings-${page}${readonlyPage(page)?' readonly':''}"><h2 id="settings-page-title" tabindex="-1">${label}</h2><p class="settings-helper">${helper}</p>${readonlyPage(page)?'<p class="settings-readonly">Read only. Saving requires signed-in send mode.</p>':''}${body}</div></div>`;
@@ -1447,6 +1447,8 @@
if (page==='appearance') return settingsShell(page,appearance(),'These choices apply in this browser only.');
if (page==='organizations') { await configRead; return settingsShell(page,window.MosaicOrgCreate.markup,'Create an organization from the template, edit it, and finish its setup. Each step names the command it equals.'); }
if (page==='model-proxy') { await configRead; return settingsShell(page,window.MosaicProxy.markup,'Accounts used by this instance, their priority, and the client keys assigned to agents.'); }
// #1873 triage-settings: Settings > Triage (triage-settings.js), the instance owner's only; anyone else gets no page.
if (page==='triage') { await configRead; if (!triageOwner || !window.MosaicTriageRoute) return notFound(); return settingsShell(page,await window.MosaicTriageRoute.section(),'The model, harness and daily token cap Triage uses. Only the instance owner sees this page.'); }
// F3: Settings > Console (console-restart.js). The server checks the owner, Origin and trail.
if (page==='console') { await configRead; await window.MosaicConsole?.ready; return settingsShell(page,window.MosaicConsole?.settingsMarkup()??'<section class="panel"><p class="small warn">The Console controls did not load. Reload the page to try again.</p></section>','Restart this Console on the same address and configuration.'); }
const s = await get.api('settings'), n = s.notifier || {};
@@ -1910,13 +1912,14 @@
return;
}
if (path === '/settings/model-proxy' && shown === route && !navigated) return;
if (path === '/settings/triage' && shown === route && !navigated) return;
if (path === '/settings/organizations' && shown?.split('?')[0] === path && !navigated) return;
if (($('agent-dialog')?.open || document.querySelector('.agent-more[aria-expanded=true]')) && !navigated) { schedule(); return; }
if (navigated || (shown !== route && !samePage)) { unmountEditor(); $('bus-view').innerHTML = loadingView(SOURCES[path.split('/')[1]]); }
let html, state = 'fresh';
const live = reader(true);
try {
if (path !== '/organization' && !wsPage(path) && !(path === '/agents' && window.MosaicOrg) && path !== '/messages' && !['/settings','/settings/appearance','/settings/model-proxy','/settings/organizations'].includes(path)) {
if (path !== '/organization' && !wsPage(path) && !(path === '/agents' && window.MosaicOrg) && path !== '/messages' && !['/settings','/settings/appearance','/settings/model-proxy','/settings/triage','/settings/organizations'].includes(path)) {
try { await live('messages'); }
catch (err) { messageCount = null; if (err.status === 403 || err.code === 'signed-out') throw err; }
}
@@ -2113,6 +2116,8 @@
canMessage = !isTester && c?.send === true;
chatSend = !isTester && c?.chatSend === true;
orgSetup = !isTester && c?.orgSetup === true;
// #1873 triage-settings: /api/config says proxyAdmin to the instance owner only, the same owner /api/triage/route serves.
triageOwner = !isTester && c?.proxyAdmin === true;
window.MosaicOrgBusStopped = !!window.MosaicOrg && c?.devOwner === true && orgSetup && c?.chatSend === false;
// The header menu offers "New organization" to the owner of a Console with organizations; nobody else can create one.
if (orgSetup && !$('org-new-link')) { const a = document.createElement('a'); a.id = 'org-new-link'; a.href = NEW_ORG; a.setAttribute('role', 'menuitem'); a.textContent = 'New organization'; document.querySelector('#avatar-menu > a[href="#/settings"]')?.after(a); }
+20 -3
View File
@@ -43,6 +43,7 @@ window.MosaicLocalTime = value => Number.isFinite(Date.parse(value)) ? new Date(
x.addEventListener('click', close);
hd.append(title, x);
const state = el('p', 'triage-state'); state.id = 'triage-state'; state.setAttribute('role', 'status');
const usage = el('p', 'note triage-usage'); usage.id = 'triage-usage'; usage.hidden = true;
const healthSec = el('details', 'triage-health-sec'); healthSec.open = true;
const healthSum = el('summary', '', 'Health');
const health = el('ul', 'triage-health'); health.id = 'triage-health';
@@ -51,10 +52,10 @@ window.MosaicLocalTime = value => Number.isFinite(Date.parse(value)) ? new Date(
const transcript = el('ol', 'triage-transcript'); transcript.id = 'triage-transcript'; transcript.tabIndex = 0;
transcript.setAttribute('aria-label', 'Triage conversation');
const compose = el('div', 'triage-compose');
drawer.append(hd, state, healthSec, convo, transcript, compose);
drawer.append(hd, state, usage, healthSec, convo, transcript, compose);
drawer.addEventListener('keydown', e => { if (e.key === 'Escape') { e.preventDefault(); close(); } });
document.body.append(drawer);
parts = { title, state, health, transcript, compose, box: null, send: null, status: null, off: null };
parts = { title, state, usage, health, transcript, compose, box: null, send: null, status: null, off: null };
}
// The message box is built once and never rewritten or removed by a read, so a draft survives every
@@ -109,6 +110,19 @@ window.MosaicLocalTime = value => Number.isFinite(Date.parse(value)) ? new Date(
};
const notRead = v => v === 'not read' || v === null || v === undefined || typeof v !== 'object';
// #1873 triage-cap: one line, today's count against the stored cap and the next reset. Only what the
// read holds: a field the read says is not set or not read is said so, never filled in.
const usageLine = view => {
if (!Object.hasOwn(view, 'usageToday')) return '';
const u = view.usageToday, c = view.cap;
if (u === 'not set' || c === 'not set') return 'Daily token cap: not set.';
if (notRead(u) || notRead(c)) return "Today's token usage: not read.";
const n = v => (Number.isSafeInteger(v) ? v.toLocaleString() : 'not read');
const reset = typeof view.resetsAt === 'string' && Number.isFinite(Date.parse(view.resetsAt)) ? ` Resets at ${when(view.resetsAt)}.` : '';
const k = Number.isSafeInteger(u.unknownTurns) && u.unknownTurns > 0 ? u.unknownTurns : 0;
const unknown = k ? ` ${k} ${k === 1 ? 'turn has' : 'turns have'} no usage reading.` : '';
return `Today: ${n(u.count)} of ${n(c.limit)} tokens.${reset}${unknown}`;
};
const configWords = c => (notRead(c) ? 'not read' : c.valid === true ? 'valid' : `not valid${c.code ? ` (${clean(c.code)})` : ''}`);
function healthLines(s) {
if (notRead(s)) return ['Health: not read'];
@@ -173,6 +187,9 @@ window.MosaicLocalTime = value => Number.isFinite(Date.parse(value)) ? new Date(
const t = parts.transcript, atBottom = t.scrollHeight - t.scrollTop - t.clientHeight < 24, top = t.scrollTop;
parts.state.textContent = stateLine(view);
parts.state.dataset.kind = view.route !== 'set' ? 'off' : view.overCapUntil ? 'cap' : view.running === true ? 'running' : 'stopped';
const usage = usageLine(view);
parts.usage.textContent = usage;
parts.usage.hidden = !usage;
const lines = healthLines(view.snapshot);
const healthKey = JSON.stringify(lines);
if (healthKey !== lastHealth) { lastHealth = healthKey; parts.health.replaceChildren(...lines.map(l => el('li', l.endsWith('not read') ? 'not-read' : '', l))); }
@@ -328,7 +345,7 @@ window.MosaicLocalTime = value => Number.isFinite(Date.parse(value)) ? new Date(
return orgOf(hash) || !seg || system(hash) ? hash : `#/orgs/${seg}${path}`;
};
// /api/workspace reads and writes across organizations (U1, U4b), so it keeps its own path.
const unscoped = /^\/api\/(?:workspace(?:\/|\?|$)|orgs(?:\/|$)|themes(?:\?|$)|org-template(?:\?|$)|org-presets(?:\?|$)|org-admin(?:\?|$)|org-validate(?:\?|$)|agents\/(?:levels|root)(?:\?|$))/;
const unscoped = /^\/api\/(?:workspace(?:\/|\?|$)|orgs(?:\/|$)|themes(?:\?|$)|org-template(?:\?|$)|org-presets(?:\?|$)|org-admin(?:\?|$)|org-validate(?:\?|$)|agents\/(?:levels|root)(?:\?|$)|triage\/route(?:\?|$))/;
window.fetch = (input, init) => {
if (typeof input !== 'string' || !input.startsWith('/api/') || unscoped.test(input)) return nativeFetch(input, init);
const send = () => nativeFetch(current ? `/api/orgs/${current}/${input.slice('/api/'.length)}` : input, init);
+1 -1
View File
@@ -7,7 +7,7 @@
<link rel="stylesheet" href="/processing.css"><script src="/processing.js" defer></script>
<link rel="stylesheet" href="/org-create.css"><script src="/org-create.js" defer></script>
<link rel="stylesheet" href="/onboarding.css"><script src="/onboarding.js" defer></script><link rel="stylesheet" href="/assistant.css">
<script src="/workspace-data.js" defer></script><script src="/workspace.js" defer></script><script src="/tasks-board.js" defer></script><script src="/console-restart.js" defer></script><script src="/display.js" defer></script><script src="/inbox.js" defer></script><script src="/files.js" defer></script><script src="/project-remote.js" defer></script><link rel="stylesheet" href="/autonomy.css"><script src="/autonomy.js" defer></script><link rel="stylesheet" href="/packs.css"><script src="/packs.js" defer></script><link rel="stylesheet" href="/workflows.css"><script src="/workflows.js" defer></script><script src="/federation.js" defer></script><script src="/brand.js" defer></script><script src="/app.js" defer></script><script src="/markdown-preview.js" defer></script><script src="/projects.js" defer></script><script src="/chat.js" defer></script><script src="/project-archive.js" defer></script><script src="/agents.js" defer></script><script src="/assistant.js" defer></script><script src="/bus.js" defer></script><script src="/requests.js" defer></script></head>
<script src="/workspace-data.js" defer></script><script src="/workspace.js" defer></script><script src="/tasks-board.js" defer></script><script src="/console-restart.js" defer></script><script src="/triage-settings.js" defer></script><script src="/display.js" defer></script><script src="/inbox.js" defer></script><script src="/files.js" defer></script><script src="/project-remote.js" defer></script><link rel="stylesheet" href="/autonomy.css"><script src="/autonomy.js" defer></script><link rel="stylesheet" href="/packs.css"><script src="/packs.js" defer></script><link rel="stylesheet" href="/workflows.css"><script src="/workflows.js" defer></script><script src="/federation.js" defer></script><script src="/brand.js" defer></script><script src="/app.js" defer></script><script src="/markdown-preview.js" defer></script><script src="/projects.js" defer></script><script src="/chat.js" defer></script><script src="/project-archive.js" defer></script><script src="/agents.js" defer></script><script src="/assistant.js" defer></script><script src="/bus.js" defer></script><script src="/requests.js" defer></script></head>
<body>
<div hidden aria-hidden="true"><select id="palette"></select><select id="mode"><option value="system">system</option><option value="light">light</option><option value="dim">dim</option><option value="dark">dark</option></select></div>
<a class="skip" href="#main">Skip to content</a>
+1
View File
@@ -832,6 +832,7 @@ html[data-organizations] .ws-drawer-acts .note{flex-basis:100%}
#triage-drawer h2:focus{outline:none}
#triage-drawer .x{margin-left:auto;flex:none;width:32px;height:32px;border:1px solid var(--line);border-radius:6px;background:var(--canvas);color:var(--strong);font-size:20px;line-height:1;cursor:pointer}
#triage-drawer .triage-state{margin:0;padding:10px 14px;flex:none;font-size:14px;font-weight:600;color:var(--strong);border-bottom:1px solid var(--line);overflow-wrap:anywhere}
#triage-drawer .triage-usage{margin:0;padding:6px 14px;flex:none;font-size:13px;color:var(--muted);border-bottom:1px solid var(--line);overflow-wrap:anywhere}
#triage-drawer .triage-state[data-kind=off],#triage-drawer .triage-state[data-kind=cap]{color:var(--warning);background:color-mix(in srgb,var(--warning) 9%,var(--surface))}
#triage-drawer .triage-state[data-kind=running]::before{content:"";display:inline-block;width:8px;height:8px;margin-right:7px;border-radius:50%;background:var(--success);vertical-align:1px}
#triage-drawer .triage-health-sec{flex:none;padding:8px 14px;border-bottom:1px solid var(--line)}
@@ -0,0 +1,133 @@
/* #1873 triage-settings: Settings > Triage, the instance owner's Triage model route. It reads and
writes /api/triage/route (triage-route-http.mjs); the server checks the owner, the Origin and Fetch
Metadata, the catalogue and the time zone, and the store keeps the revision and the receipt. This
file only asks and reports. bus.js offers the page only when /api/config says proxyAdmin, which is
the same owner the route serves. The fields live here, so a repaint keeps what the owner typed. */
(() => {
'use strict';
const $ = id => document.getElementById(id);
const esc = s => String(s).replace(/[&<>"']/g, c => ({ '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;' })[c]);
const PATH = '/api/triage/route';
// The time zones this browser knows, offered as suggestions only; the server decides.
const zones = (() => { try { return Intl.supportedValuesOf('timeZone'); } catch { return []; } })();
// draft holds the typed fields; base is the revision they were read at, which a save sends, so a
// change saved elsewhere since then answers as stale instead of being written over.
let data = null, readLine = '', draft = null, base = 0, busy = false, line = '', conflict = false;
const call = init => window.fetch(PATH, { cache: 'no-store', redirect: 'manual', ...init });
// The stored route's fields, or empty ones: there is no default model, harness, cap or zone.
const fromRoute = r => ({ model: r?.model ?? '', harness: r?.harness ?? '', cap: r ? String(r.cap) : '', timeZone: r?.timeZone ?? '' });
async function read() {
readLine = '';
let r, a = null;
try { r = await call({ method: 'GET' }); } catch { data = null; readLine = 'The Triage route could not be read. Reload the page to try again.'; return; }
try { a = await r.json(); } catch { /* refused below */ }
if (r.type === 'opaqueredirect') { data = null; readLine = 'Sign in again, then reload.'; return; }
if (!r.ok || !a || typeof a !== 'object') { data = null; readLine = typeof a?.message === 'string' ? a.message : 'The Triage route could not be read.'; return; }
// An untouched page follows the store; typed fields stay, with the revision they were read at.
const untouched = draft === null || JSON.stringify(draft) === JSON.stringify(fromRoute(data?.route ?? null));
data = a;
if (untouched) { draft = fromRoute(a.route); base = Number.isSafeInteger(a.revision) ? a.revision : 0; }
}
const models = () => (Array.isArray(data?.choices?.models) ? data.choices.models : []);
const harnessesOf = key => models().find(m => m.key === key)?.harnesses ?? [];
function stored() {
const r = data?.route;
if (!r) return `<p id="triage-route-stored"><strong>Not set.</strong> ${esc(data?.routeMessage ?? 'No Triage route is set.')}</p>`;
return `<p id="triage-route-stored">Stored route: <strong>${esc(r.model)}</strong> on <strong>${esc(r.harness)}</strong>, cap ${esc(Number(r.cap).toLocaleString('en-US'))} tokens a day, reset ${esc(r.reset)} ${esc(r.timeZone)}. Revision ${esc(r.revision)}, saved ${esc(r.updatedAt)}.</p>`;
}
function form() {
const d = draft ?? fromRoute(null);
const send = data?.send === true;
const off = busy || !send ? ' disabled' : '';
const list = models();
const offered = list.some(m => m.key === d.model);
const modelOptions = `<option value=""${d.model === '' ? ' selected' : ''}>Choose a model</option>`
+ (d.model && !offered ? `<option value="${esc(d.model)}" selected>${esc(d.model)} (not offered now)</option>` : '')
+ list.map(m => `<option value="${esc(m.key)}"${m.key === d.model ? ' selected' : ''}${m.available ? '' : ' disabled'}>${esc(m.label && m.label !== m.key ? `${m.label} (${m.key})` : m.key)}${m.available ? '' : ' (unavailable)'}</option>`).join('');
const hs = harnessesOf(d.model);
const harnessOptions = `<option value=""${d.harness === '' ? ' selected' : ''}>${d.model ? 'Choose a harness' : 'Choose a model first'}</option>`
+ (d.harness && !hs.includes(d.harness) ? `<option value="${esc(d.harness)}" selected>${esc(d.harness)} (not offered for this model)</option>` : '')
+ hs.map(h => `<option value="${esc(h)}"${h === d.harness ? ' selected' : ''}>${esc(h)}</option>`).join('');
const status = data?.choices?.status;
const catalogue = status === 'ready' ? '' : `<p class="small warn" id="triage-route-catalogue">No organization bus offered a model catalogue (${esc(status ?? 'unknown')}), so no route can be saved now.</p>`;
const conflictButton = conflict ? ' <button type="button" class="btn" id="triage-route-reload">Reload route</button>' : '';
return `${catalogue}<form id="triage-route-form" novalidate><div class="filters">`
+ `<label>Model <select id="triage-route-model"${off}>${modelOptions}</select></label>`
+ `<label>Harness <select id="triage-route-harness"${off}>${harnessOptions}</select></label>`
+ `<label>Daily cap, tokens <input id="triage-route-cap" type="number" min="1" step="1" inputmode="numeric" value="${esc(d.cap)}"${off}></label>`
+ `<label>Reset <input id="triage-route-reset" value="00:00" readonly aria-readonly="true"></label>`
+ `<label>Time zone <input id="triage-route-zone" list="triage-route-zones" autocomplete="off" spellcheck="false" placeholder="Region/City" value="${esc(d.timeZone)}"${off}></label>`
+ `</div><datalist id="triage-route-zones">${zones.map(z => `<option value="${esc(z)}"></option>`).join('')}</datalist>`
+ `<div class="settings-actions"><button type="submit" id="triage-route-save"${off}>${busy ? 'Saving…' : 'Save'}</button>${conflictButton} <span role="status" id="triage-route-line" tabindex="-1">${esc(line)}</span></div>`
+ `</form>${send ? '' : '<p class="small muted" id="triage-route-readonly">This Console is read-only, so the route cannot be saved here.</p>'}`;
}
function markup() {
const body = data === null
? `<p class="small warn">${esc(readLine || 'The Triage route could not be read.')}</p><p role="status" id="triage-route-line" tabindex="-1">${esc(line)}</p>`
: `${stored()}${form()}<p class="small muted" id="triage-route-note">${esc(data.note ?? 'Saving a route does not start Triage. The key and the wiring are separate.')}</p>`;
return `<section class="panel" id="triage-route" aria-labelledby="triage-route-title"><h3 id="triage-route-title">Triage model route</h3>${body}</section>`;
}
const repaint = () => { const panel = $('triage-route'); if (panel) panel.outerHTML = markup(); };
// Called by bus.js when the page draws: a fresh read each time, the typed fields kept.
async function section() { await read(); return markup(); }
async function save() {
if (busy || data?.send !== true) return;
const d = draft ?? fromRoute(null);
const cap = /^\d+$/.test(d.cap.trim()) ? Number(d.cap.trim()) : d.cap.trim() === '' ? null : Number.NaN;
const body = { model: d.model, harness: d.harness, cap: Number.isFinite(cap) ? cap : d.cap, timeZone: d.timeZone.trim(), revision: base };
busy = true; conflict = false; line = 'Saving…'; repaint();
let r, a = null;
try { r = await call({ method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(body) }); }
catch { busy = false; line = 'Outcome unknown. Reload the route to see whether it was saved.'; conflict = true; repaint(); return; }
try { a = await r.json(); } catch { /* refused below */ }
busy = false;
if (r.ok && a?.route) {
data = { ...data, route: a.route, revision: a.revision, routeMessage: undefined };
draft = fromRoute(a.route); base = a.revision;
line = `Saved. Revision ${a.revision}.`;
repaint();
$('triage-route-line')?.focus?.();
return;
}
if (r.type === 'opaqueredirect') line = 'Sign in again, then retry.';
else line = typeof a?.message === 'string' ? a.message : 'The route was refused. Nothing was saved.';
if (r.status === 409 && a?.error === 'triage-route-stale') conflict = true;
repaint();
$(conflict ? 'triage-route-reload' : 'triage-route-save')?.focus();
}
async function reload() {
if (busy) return;
draft = null; conflict = false; line = '';
await read();
line = data ? 'Route read again. Your fields now show the stored route.' : '';
repaint();
}
const field = { 'triage-route-model': 'model', 'triage-route-harness': 'harness', 'triage-route-cap': 'cap', 'triage-route-zone': 'timeZone' };
document.addEventListener('input', e => {
const k = field[e.target?.id]; if (!k || !$('triage-route')) return;
draft = { ...(draft ?? fromRoute(null)), [k]: e.target.value };
});
document.addEventListener('change', e => {
if (e.target?.id !== 'triage-route-model') return;
// A new model narrows the harness list; a harness the model does not take is cleared.
const d = draft = { ...(draft ?? fromRoute(null)), model: e.target.value };
const hs = harnessesOf(d.model);
if (!hs.includes(d.harness)) d.harness = hs.length === 1 ? hs[0] : '';
repaint(); $('triage-route-model')?.focus();
});
document.addEventListener('submit', e => { if (e.target?.id !== 'triage-route-form') return; e.preventDefault(); save(); });
document.addEventListener('click', e => { if (e.target?.id === 'triage-route-reload') reload(); });
const hasDraft = () => !!(draft && data && JSON.stringify(draft) !== JSON.stringify(fromRoute(data.route)));
window.MosaicTriageRoute = { section, hasDraft };
})();
+25 -17
View File
@@ -1117,16 +1117,21 @@
};
const DRAWER_LIMIT = 5, STOP_REASON = 'Stopped from the agent drawer.';
const STOP_UNKNOWN = 'Outcome unknown. Read the agents again before trying; nothing was retried.';
// The Stop state of the drawer's agent survives repaints: `confirm` while it asks, `busy` while the
// SP20: an organization that refuses a Stop or a Restart, in the same line for both.
const orgRefusal = (e, orgName) => e === 'org-archived' ? `${orgName} is archived, so its agents are not changed. Nothing changed.`
: e === 'org-unavailable' ? `${orgName} is not available to this Console. Nothing changed.` : '';
// Each drawer agent's question and Stop survive repaints: `confirm` while it asks, `busy` while the
// request is open, then the result in words beside the button. SP19: `kind` says whether Stop or
// Restart asks (no kind is Stop). While either asks or runs, both buttons wait.
let stopState = null; // { id, kind, confirm, busy, text }
// Restart asks (no kind is Stop). While either asks or runs, both buttons wait. SP20: kept per agent,
// as restarts are, so two Stops in two drawers each keep their line. An entry goes when the drawer
// closes after it finished, when its question is cancelled, or when its Restart starts.
const stops = new Map(); // id -> { kind, confirm, busy, text }
// SP19: each drawer agent's restart, kept per agent so several can run at once and none loses its
// line when another drawer asks or stops: `busy` while it runs, then its outcome in words. An entry
// goes when a new question starts on that agent, or when the drawer closes after it finished.
const restarts = new Map(); // id -> { busy, text }
// The state a drawer draws: its own question or Stop first, else its restart.
const actState = id => stopState?.id === id ? stopState : restarts.has(id) ? { id, kind: 'restart', ...restarts.get(id) } : null;
const actState = id => stops.has(id) ? { id, ...stops.get(id) } : restarts.has(id) ? { id, kind: 'restart', ...restarts.get(id) } : null;
const RESTART_REASON = 'restart from the agent drawer';
let drawerFocus = false, drawerShown = null;
// The drawer holds at most one history entry. A row opened from the page without a drawer pushes one
@@ -1396,7 +1401,7 @@
// Closing goes Back when a row link on this page pushed the current entry; otherwise it drops
// `agent` from the address in place. Either way the page repaints without it and focus returns to the row.
function closeDrawer() {
if (!stopState?.busy) stopState = null;
for (const [k, st] of stops) if (!st.busy) stops.delete(k);
for (const [k, r] of restarts) if (!r.busy) restarts.delete(k);
if (stateOf()?.[DRAWER_MARK] && queryOf().has('agent')) { history.back(); return; }
history.replaceState(unmarked(), '', sansAgent());
@@ -1433,10 +1438,11 @@
}
async function stopAgent(d) {
const id = d.dataset.wsDrawer, org = d.dataset.drawerOrgId, name = id.slice(org.length + 1), orgName = d.dataset.drawerOrgName;
stopState = { id, busy: true, text: `Stopping ${name}…` };
stops.set(id, { busy: true, text: `Stopping ${name}…` });
const repaint = () => document.dispatchEvent(new CustomEvent('mosaic-workspace-render'));
const paint = () => { const now = document.querySelector(`[data-ws-drawer="${CSS.escape(id)}"]`); if (!now) return; const s = now.querySelector('[data-drawer-status]'); if (s) s.textContent = stopState.text; const b = now.querySelector('[data-drawer-stop]'); if (b) { b.disabled = !!stopState.busy; b.firstChild.textContent = stopState.busy ? 'Stopping…' : 'Stop'; } const rb = now.querySelector('[data-drawer-restart]'); if (rb) rb.disabled = !!stopState.busy; now.querySelector('[data-drawer-stop-confirm]')?.remove(); };
const finish = text => { stopState = { id, text }; paint(); const b = document.querySelector(`[data-ws-drawer="${CSS.escape(id)}"] [data-drawer-stop]`); b?.focus({ preventScroll: true }); };
// Paint this agent's drawer in place, when it is on screen.
const paint = () => { const now = document.querySelector(`[data-ws-drawer="${CSS.escape(id)}"]`), st = stops.get(id); if (!now || !st) return; const s = now.querySelector('[data-drawer-status]'); if (s) s.textContent = st.text; const b = now.querySelector('[data-drawer-stop]'); if (b) { b.disabled = !!st.busy; b.firstChild.textContent = st.busy ? 'Stopping…' : 'Stop'; } const rb = now.querySelector('[data-drawer-restart]'); if (rb) rb.disabled = !!st.busy; now.querySelector('[data-drawer-stop-confirm]')?.remove(); };
const finish = text => { stops.set(id, { text }); paint(); const b = document.querySelector(`[data-ws-drawer="${CSS.escape(id)}"] [data-drawer-stop]`); b?.focus({ preventScroll: true }); };
paint();
let res, body = null;
try { res = await fetch(`/api/orgs/${enc(org)}/agent/terminate`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ instance: name, reason: STOP_REASON }), cache: 'no-store', redirect: 'manual' }); }
@@ -1444,6 +1450,8 @@
if (res.type === 'opaqueredirect' || res.status === 401) return finish('Not stopped: sign in again, then read the agents.');
try { body = await res.json(); } catch { body = null; }
if (!res.ok || body?.sent !== true && !body?.request) {
const orgWhy = orgRefusal(body?.error, orgName);
if (orgWhy) return finish(`Not stopped: ${orgWhy}`);
if (body?.sent === false && typeof body.message === 'string' && body.message) return finish(`Not stopped: ${body.message}`);
if (res.status === 403 && body?.error === 'read-only') return finish(`Not stopped: this Console only reads agents in ${orgName} (read-only).`);
if (body?.sent === false && typeof body.error === 'string') return finish(`Not stopped: the bus refused (${body.error}).`);
@@ -1451,14 +1459,14 @@
}
const request = typeof body?.request === 'string' ? body.request : '';
if (!/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/.test(request)) return finish(STOP_UNKNOWN);
stopState = { id, busy: true, text: `Stop asked: request ${request} is open.` }; paint();
stops.set(id, { busy: true, text: `Stop asked: request ${request} is open.` }); paint();
for (let tries = 0; tries < 90; tries++) {
await new Promise(r => setTimeout(r, 2000));
let r = null;
try { const got = await fetch(`/api/orgs/${enc(org)}/agent/request?id=${enc(request)}`, { cache: 'no-store', redirect: 'manual' }); if (got.type === 'opaqueredirect') return finish('Sign in again, then read the agents.'); r = got.ok ? await got.json() : null; } catch { continue; }
if (r?.request === request && r.state === 'closed') {
if (r.reason) return finish(`Not stopped: ${r.reason}.`);
stopState = { id, text: `Stopped ${name}.` }; repaint(); return;
stops.set(id, { text: `Stopped ${name}.` }); repaint(); return;
}
}
finish(`Request ${request} is still open. Read the agents again later.`);
@@ -1472,8 +1480,8 @@
if (d?.status === 403 && e === 'read-only') return `this Console only reads agents in ${orgName} (read-only).`;
if (d?.status === 403 && (e === 'unavailable' || e === 'owner-only')) return `this Console is not the owner of ${orgName}, so it does not restart its agents.`;
if (e === 'no-bus-host') return `the bus of ${orgName} is not running. Nothing changed.`;
if (e === 'org-archived') return `${orgName} is archived, so its agents are not changed. Nothing changed.`;
if (e === 'org-unavailable') return `${orgName} is not available to this Console. Nothing changed.`;
const orgWhy = orgRefusal(e, orgName);
if (orgWhy) return orgWhy;
if (d?.result?.sent === false && typeof m === 'string' && m) return m;
if (d?.result?.sent === false && typeof e === 'string') return `the bus refused (${e}).`;
return STOP_UNKNOWN;
@@ -1501,11 +1509,11 @@
const thread = d.querySelector('[data-drawer-restart]')?.dataset.drawerThread === 'yes';
if (restarts.get(id)?.busy) return;
// The question is answered: from now on this agent's drawer draws its restart.
if (stopState?.id === id) stopState = null;
stops.delete(id);
restarts.set(id, { busy: true, text: '' });
// Paint this agent's drawer in place, when it is on screen and no question of its own holds it.
const paint = () => {
const now = document.querySelector(`[data-ws-drawer="${CSS.escape(id)}"]`), r = restarts.get(id); if (!now || !r || stopState?.id === id) return;
const now = document.querySelector(`[data-ws-drawer="${CSS.escape(id)}"]`), r = restarts.get(id); if (!now || !r || stops.has(id)) return;
const s = now.querySelector('[data-drawer-status]'); if (s) s.textContent = r.text;
const rb = now.querySelector('[data-drawer-restart]'), sb = now.querySelector('[data-drawer-stop]');
if (rb) { rb.disabled = r.busy; rb.firstChild.textContent = r.busy ? 'Restarting…' : 'Restart'; }
@@ -1548,11 +1556,11 @@
if (e.target.closest('[data-drawer-close]')) { closeDrawer(); return; }
const id = d.dataset.wsDrawer;
// SP19: nothing starts while this agent's Stop or Restart runs.
const held = (stopState?.id === id && stopState.busy) || !!restarts.get(id)?.busy;
const held = !!stops.get(id)?.busy || !!restarts.get(id)?.busy;
const ask = e.target.closest('[data-drawer-stop]') ? 'stop' : e.target.closest('[data-drawer-restart]') ? 'restart' : null;
if (ask && !held) {
const b = d.querySelector(`[data-drawer-${ask}]`), name = d.dataset.drawerName, who = `${name} in ${d.dataset.drawerOrgName}`;
stopState = { id, kind: ask, confirm: true, text: '' }; restarts.delete(id);
stops.set(id, { kind: ask, confirm: true, text: '' }); restarts.delete(id);
d.querySelector('.ws-drawer-confirm')?.remove();
b.insertAdjacentHTML('afterend', ask === 'stop' ? stopConfirm(name, who) : restartConfirm(name, who, b.dataset.drawerThread === 'yes'));
for (const x of d.querySelectorAll('[data-drawer-stop], [data-drawer-restart]')) x.disabled = true;
@@ -1569,7 +1577,7 @@
// Cancel the Stop or Restart question: nothing sent, both buttons enabled again, focus on the one that asked.
function cancelStop(d) {
const kind = d.querySelector('[data-drawer-restart-confirm]') ? 'restart' : 'stop';
stopState = null;
stops.delete(d.dataset.wsDrawer);
d.querySelector('.ws-drawer-confirm')?.remove();
for (const x of d.querySelectorAll('[data-drawer-stop], [data-drawer-restart]')) x.disabled = false;
d.querySelector(`[data-drawer-${kind}]`)?.focus({ preventScroll: true });
+1 -1
View File
@@ -66,7 +66,7 @@ const files = new Map([
['/', ['index.html', 'text/html; charset=utf-8']],
['/proxy.js', ['proxy.js', 'text/javascript; charset=utf-8']],
['/project-remote.js', ['project-remote.js', 'text/javascript; charset=utf-8']],
...['app.js', 'brand.js', 'bus.js', 'markdown-preview.js', 'chat.js', 'project-archive.js', 'processing.js', 'agents.js', 'federation.js', 'requests.js', 'org-create.js', 'console-restart.js', 'onboarding.js', 'workspace-data.js', 'workspace.js', 'tasks-board.js', 'assistant.js'].map(f => ['/' + f, [f, 'text/javascript; charset=utf-8']]),
...['app.js', 'brand.js', 'bus.js', 'markdown-preview.js', 'chat.js', 'project-archive.js', 'processing.js', 'agents.js', 'federation.js', 'requests.js', 'org-create.js', 'console-restart.js', 'onboarding.js', 'workspace-data.js', 'workspace.js', 'tasks-board.js', 'assistant.js', 'triage-settings.js'].map(f => ['/' + f, [f, 'text/javascript; charset=utf-8']]),
['/files.js', ['files.js', 'text/javascript; charset=utf-8']],
['/projects.js', ['projects.js', 'text/javascript; charset=utf-8']],
['/projects.css', ['projects.css', 'text/css; charset=utf-8']],
+164 -8
View File
@@ -32,6 +32,17 @@
// directory and HOME (HOME not counted against the stack's own code), here and again in
// hostedBinds. A refused answer reaches no spawn, answers triage-route-invalid, and writes
// ROUTE_REFUSED to the transcript; a refusal at the binds also removes what the start wrote.
//
// #1873 triage-cap: the relay counts each turn. usage.begin takes a baseline of the harness's
// transcript with the transcript usage-source the bus uses; usage.end takes the delta (the bus's
// own rules, triage-usage.mjs) and appends one line to <dataRoot>/triage/usage.jsonl. When a cap is
// stored (triage-route-store.mjs), the daily-quota plugin decides over the lines in that file: before
// a start (admit), before the runner takes queued messages (message.receive) and before each
// message's turn (the runner's state read outside a turn). Over the cap, or with the store or the
// file unreadable, the turn in flight finishes, every message still queued gets a host line and
// leaves the queue, and the runner is told its session is gone. Nothing is held in memory: each
// check reads the file again, so a restart counts what the file holds. With no cap stored, nothing
// is refused for usage and the lines are still written.
import { spawn } from "node:child_process";
import { randomBytes, timingSafeEqual } from "node:crypto";
@@ -48,6 +59,9 @@ import { builtinTools } from "../../harness/src/bundle.mjs";
import { FOUNDER_ENV } from "../../harness/src/runner.mjs";
import { extractCards, plainText } from "./triage-cards.mjs";
import { SNAPSHOT_FILE, writeSnapshot } from "./triage-snapshot.mjs";
import { readRoute } from "./triage-route-store.mjs";
import { ID as USAGE_ID, appendUsage, capState, delta, nextReset, readUsage, snapshot, usagePath } from "./triage-usage.mjs";
import { readTurn } from "../../plugins/usage-source/transcript/index.mjs";
export const RUNNER = RUNNER_FILES[0];
export const RELAY_REFUSED = "triage-refused";
@@ -58,6 +72,8 @@ export const UNANSWERED = "Triage stopped before it answered this message. Send
export const NOT_STARTED = "Triage did not start, so this message was not delivered.";
export const NOT_TAKEN = "Triage stopped before it read this message. Send it again.";
export const ROUTE_REFUSED = "Triage did not start: the model route's answer did not pass the host's checks, so this message was not delivered.";
export const overCapNotTaken = (until) => `Triage reached its daily token cap before it took this message. Send it again after ${until}.`;
export const USAGE_NOT_READ = "Triage could not read its token usage or its cap, so it did not take this message.";
// An over-cap answer is held for at most this long; a later time is not taken.
export const CAP_HOLD_MAX_MS = 48 * 3600 * 1000;
const ISO_TIME = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d{1,3})?Z$/;
@@ -179,7 +195,10 @@ function sameCap(a, b) {
// The relay server. `onSend({ body, in_reply_to, result })` keeps a human-bound send, `result` true
// only for class RESULT, the class the runner gives its answer to a turn; `take()` returns the
// queued owner messages; `read(id)` drops one that was answered.
export function createRelay({ socket, cap, take, read, onSend, log = () => {} }) {
// #1873 triage-cap: `usage(phase, turn)` (a promise) counts the turn; the relay answers usage.begin
// and usage.end only once it settles. `stopped(verb)` is asked before state and message.receive;
// true answers "not-holder", which ends the runner's loop after the turn in flight.
export function createRelay({ socket, cap, take, read, onSend, usage = null, stopped = () => false, log = () => {} }) {
const answer = (conn, value) => { try { conn.end(JSON.stringify(value) + "\n"); } catch {} };
const refuse = (conn) => answer(conn, { ok: false, error: RELAY_REFUSED, reason: "the Triage host answers only a message to the person" });
const server = createServer((conn) => {
@@ -200,6 +219,13 @@ export function createRelay({ socket, cap, take, read, onSend, log = () => {} })
const args = req.args && typeof req.args === "object" && !Array.isArray(req.args) ? req.args : {};
const verb = req.verb;
try {
if (usage && (verb === "usage.begin" || verb === "usage.end")) {
return void Promise.resolve()
.then(() => usage(verb === "usage.begin" ? "begin" : "end", args.turn))
.catch((e) => log(`triage relay: ${verb} not counted: ${e?.code ?? "error"}`))
.then(() => answer(conn, { ok: true, result: {} }));
}
if ((verb === "state" || verb === "message.receive") && stopped(verb)) return answer(conn, { ok: false, error: "not-holder" });
if (Object.hasOwn(LIFECYCLE, verb)) return answer(conn, { ok: true, result: LIFECYCLE[verb]() });
if (verb === "message.receive") return answer(conn, { ok: true, result: take() });
if (verb === "message.read") { read(args.id); return answer(conn, { ok: true, result: {} }); }
@@ -235,11 +261,13 @@ export function createRelay({ socket, cap, take, read, onSend, log = () => {} })
// harnessModule async (name) -> the harness module (envNames, installRoots) or null; the
// repository's packages/plugins/harness/<name> by default
// home HOME, which no mount may reach outside the stack's own code; the Console user's by default
export function createTriageHost({ dataRoot, route = null, readSnapshot = async () => null, env = process.env, node = process.execPath, bwrap = BWRAP, now = () => new Date(), devOwner = false, log = () => {}, roleDir = join(REPO, "roles"), readOnly = [], pollInterval = null, proxyPort = null, tokenPattern = null, harnessModule = repoHarnessModule, home = homedir() }) {
// usageSource { readTurn({ harness, root }) } the transcript usage-source; the repository's own by default
export function createTriageHost({ dataRoot, route = null, readSnapshot = async () => null, env = process.env, node = process.execPath, bwrap = BWRAP, now = () => new Date(), devOwner = false, log = () => {}, roleDir = join(REPO, "roles"), readOnly = [], pollInterval = null, proxyPort = null, tokenPattern = null, harnessModule = repoHarnessModule, home = homedir(), usageSource = { readTurn } }) {
if (typeof dataRoot !== "string" || !dataRoot.startsWith("/")) throw new TriageError("triage-no-data-root", "no data root");
if (proxyPort !== null && !(Number.isInteger(proxyPort) && proxyPort > 0 && proxyPort <= 65535)) throw new TypeError("proxyPort must be a port number or null");
if (tokenPattern !== null && !(tokenPattern instanceof RegExp)) throw new TypeError("tokenPattern must be a RegExp or null");
if (typeof harnessModule !== "function") throw new TypeError("harnessModule must be a function");
if (typeof usageSource?.readTurn !== "function") throw new TypeError("usageSource must have readTurn");
const tokenShape = tokenPattern && new RegExp(`^(?:${tokenPattern.source})$`, tokenPattern.flags.replace(/[gmy]/g, ""));
// What no mount source may lie in or around, besides the run directory and the fixed list. HOME
// does not refuse the stack's own code when the stack runs from under it (a copy or a worktree).
@@ -257,6 +285,10 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
let cap = null;
let overCapUntil = null; // the route's over-cap answer, held until its time passes
const queue = [];
let usage = null; // this run's counting: { run, root, harness, baselines, ended, inTurn }
let usageChain = Promise.resolve(); // usage calls and exit records, one at a time
let usageLost = false; // a turn's line could not be written: the count is short, so a stored cap refuses
const badSeen = new Set();
const ensureDirs = () => {
mkdirSync(dataRoot, { recursive: true, mode: 0o700 });
@@ -320,14 +352,16 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
writeFileSync(bundle.prompt, hostPrompt(contract, { snapshotPath: join(workspace, "health", SNAPSHOT_FILE), steps }), { mode: 0o600, flag: "wx" });
writeFileSync(bundle.policy, JSON.stringify(policy, null, 2) + "\n", { mode: 0o600, flag: "wx" });
writeFileSync(bundle.tools, JSON.stringify(tools, null, 2) + "\n", { mode: 0o600, flag: "wx" });
const run = `triage-${now().toISOString().replace(/[^0-9]/g, "").slice(0, 14)}`;
const session = {
bundle, adapter: admitted.adapter, harness: admitted.harness, harnessEnv: admitted.harnessEnv, turnEnd: admitted.turnEnd,
workspace, sessionDir: join(a, "session"), toolSocket: join(a, "tools.sock"), turnMarker: join(a, "turn.done"),
brokerSocket: join(a, "relay.sock"), business: "system", instance: "triage", run: `triage-${now().toISOString().replace(/[^0-9]/g, "").slice(0, 14)}`,
brokerSocket: join(a, "relay.sock"), business: "system", instance: "triage", run,
model: admitted.model, provider: admitted.provider ?? "",
...(Number.isInteger(pollInterval) && pollInterval >= 100 ? { pollInterval } : {}),
};
writeFileSync(join(a, "session.json"), JSON.stringify(session, null, 2) + "\n", { mode: 0o600, flag: "wx" });
return run;
};
// The route's answer, checked. It returns a copy taken once, so a getter or a later change to
@@ -363,13 +397,114 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
if (e?.code === "hosted-binds-refused") refuse(e.fence);
throw e;
}
return { admitted: a, installRoots };
// Where the harness keeps its session files, as the bus binds it (the module's transcript()).
// Only a path inside the run directory is read; anything else reads as no transcript.
let transcriptRoot = null;
try {
const r = typeof mod.transcript === "function" ? mod.transcript({ runDir: paths.agent }) : null;
if (typeof r === "string" && r.startsWith(paths.agent + "/") && !r.split("/").includes("..")) transcriptRoot = r;
} catch { transcriptRoot = null; }
return { admitted: a, installRoots, transcriptRoot };
};
// #1873 triage-cap. A baseline or a reading of the transcript, with the bus's codes.
const sample = async (u) => {
if (u.root === null) return { status: "unknown", code: "no-transcript" };
try { return snapshot(await usageSource.readTurn({ harness: u.harness, root: u.root })); }
catch { return { status: "unknown", code: "source-failed" }; }
};
// A turn id the runner names, or "unnamed". The first begin for a turn wins, as on the bus.
const turnKey = (turn) => (typeof turn === "string" && USAGE_ID.test(turn) ? turn : null);
const usageBegin = (u, turn) => {
const named = turnKey(turn);
if (!named) log("triage host: usage.begin without a usable turn id");
const k = named ?? "unnamed";
u.inTurn = true;
if (!u.baselines.has(k)) u.baselines.set(k, sample(u));
};
// One line per turn: the delta since its baseline, known or unknown.
const finishTurn = async (u, k) => {
const base = u.baselines.has(k) ? await u.baselines.get(k) : null;
u.baselines.delete(k);
const result = delta(base, await sample(u));
const known = result.status === "known";
if (!known) log(`triage host: turn usage unknown: ${result.code}`);
const line = { at: now().toISOString(), run: u.run, turn: k, status: result.status,
input: known ? result.input : null, output: known ? result.output : null, cacheRead: known ? result.cacheRead : null, cacheWrite: known ? result.cacheWrite : null };
try {
appendUsage(usagePath(dataRoot), line);
} catch (e) {
usageLost = true;
log(`triage host: usage line not written: ${e?.code ?? "error"}`);
}
};
const usageEnd = async (u, turn) => {
u.inTurn = false;
const named = turnKey(turn);
if (!named) log("triage host: usage.end without a usable turn id");
if (named && u.ended.has(named)) { log("triage host: usage.end repeated for a turn; not counted twice"); return; }
if (named) u.ended.add(named);
await finishTurn(u, named ?? "unnamed");
};
const chainUsage = (fn) => (usageChain = usageChain.then(fn).catch((e) => log(`triage host: usage failed: ${e?.code ?? "error"}`)));
// The stored cap's state, read from the store and the usage file each time.
// { state: "not set" } no cap stored
// { state: "route-unreadable" } | { state: "usage-unreadable", cap, resetsAt? } refused
// { state: "under" | "over", cap, usageToday, resetsAt }
const capNow = () => {
let stored;
try { stored = readRoute(dataRoot); } catch { return { state: "route-unreadable" }; }
if (!stored) return { state: "not set" };
const at = now().getTime();
let resetsAt;
try { resetsAt = new Date(nextReset(stored.cap, at)).toISOString(); } catch { return { state: "usage-unreadable", cap: stored.cap }; }
if (usageLost) return { state: "usage-unreadable", cap: stored.cap, resetsAt };
let read;
try { read = readUsage(usagePath(dataRoot)); } catch { return { state: "usage-unreadable", cap: stored.cap, resetsAt }; }
for (const b of read.bad) {
const key = `${read.ino}:${b.line}`;
if (badSeen.has(key)) continue;
badSeen.add(key);
log(`triage host: usage.jsonl line ${b.line} is not a usage line; it counts as an unknown turn`);
}
try {
const s = capState({ cap: stored.cap, records: read.records, now: at });
return { state: s.allow ? "under" : "over", cap: stored.cap, usageToday: s.usageToday, resetsAt: s.resetsAt };
} catch { return { state: "usage-unreadable", cap: stored.cap, resetsAt }; }
};
const unsafeRoot = () => {
let st;
try { st = lstatSync(paths.root); } catch { return false; }
return st.isSymbolicLink() || !st.isDirectory() || (typeof process.getuid === "function" && st.uid !== process.getuid());
};
const refusedByCap = (c) => c.state === "over" || c.state === "usage-unreadable" || c.state === "route-unreadable";
// Before the runner takes messages or starts a message's turn: over the cap, every message still
// queued gets a host line and leaves the queue, and the runner's loop ends. That holds with nothing
// queued too: an idle session over the cap ends at its next receive, and the next message is
// refused by admit. A state read inside a turn (a continuation) is not stopped: the turn finishes.
const capStop = (verb) => {
const u = usage;
if (verb === "state" && u?.inTurn) return false;
const c = capNow();
if (!refusedByCap(c)) return false;
log(`triage host: stopping the session at ${verb}: ${c.state === "over" ? "over the daily token cap" : "usage or cap not read"}`);
// A message whose turn began and was never answered says so; one never begun says why not.
const begun = (id) => Boolean(u && (u.baselines.has(id) || u.ended.has(id)));
for (const m of queue.splice(0)) {
const text = begun(m.id) ? UNANSWERED : c.state === "over" ? overCapNotTaken(c.resetsAt) : USAGE_NOT_READ;
try { append({ at: now().toISOString(), from: "host", in_reply_to: m.id, text }); }
catch { log("triage host: transcript write failed"); }
}
return true;
};
const start = async (answer) => {
const { admitted, installRoots } = await checkAdmitted(answer);
const { admitted, installRoots, transcriptRoot } = await checkAdmitted(answer);
ensureDirs();
writeBundle(admitted);
const run = writeBundle(admitted);
const u = { run, root: transcriptRoot, harness: admitted.harness, baselines: new Map(), ended: new Set(), inTurn: false };
usage = u;
cap = randomBytes(32).toString("hex");
relay = await createRelay({
socket: join(paths.agent, "relay.sock"), cap,
@@ -393,6 +528,8 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
const i = in_reply_to && result === true ? queue.findIndex((m) => m.id === in_reply_to) : -1;
if (i >= 0) queue.splice(i, 1);
},
usage: (phase, turn) => chainUsage(() => (phase === "begin" ? usageBegin(u, turn) : usageEnd(u, turn))),
stopped: capStop,
log,
});
let list;
@@ -427,6 +564,9 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
if (child !== mine) return;
lastExit = { code: Number.isInteger(code) ? code : null, signal: signal ?? null, at: now().toISOString() };
child = null;
// A turn the session began and never ended (killed, crashed) still spent tokens: it gets its line.
// The open turns are read when this runs, after any usage.end already in the chain.
chainUsage(async () => { for (const k of [...u.baselines.keys()]) await finishTurn(u, k); });
// Every message still queued leaves it, so none holds a slot with no session to answer it, and
// the transcript says which: one the session took and never answered, or one it never read.
for (const m of queue.splice(0)) {
@@ -456,6 +596,15 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
if (!route || typeof route.admit !== "function") return { ok: false, code: "triage-no-route" };
const held = capUntil();
if (held !== null) return { ok: false, code: "triage-over-cap", until: held };
// A turn still being counted (a session that just ended mid-turn, or a usage.end in flight) is in
// the file before the count is read.
await usageChain;
const c = capNow();
// The store sits in the Triage directory: one that is a link or not the Console user's is the
// unsafe directory start would refuse, not an unreadable route.
if (c.state === "route-unreadable") return { ok: false, code: unsafeRoot() ? "triage-dir-unsafe" : "triage-route-unreadable" };
if (c.state === "usage-unreadable") return { ok: false, code: "triage-usage-unreadable" };
if (c.state === "over") return { ok: false, code: "triage-over-cap", until: c.resetsAt };
let a;
try { a = await route.admit(); } catch { return { ok: false, code: "triage-route-unreadable" }; }
if (a?.ok === true) return a;
@@ -474,9 +623,15 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
get pid() { return child?.pid ?? null; },
routeSet: () => Boolean(route && typeof route.admit === "function"),
async state() {
const c = capNow();
const unset = c.state === "not set";
return {
route: this.routeSet() ? "set" : "not set",
overCapUntil: capUntil(),
overCapUntil: c.state === "over" ? c.resetsAt : capUntil(),
// #1873 triage-cap: today's count against the stored cap. Names and numbers only.
usageToday: unset ? "not set" : c.usageToday ?? "not read",
cap: unset ? "not set" : c.cap ? { limit: c.cap.limit, reset: c.cap.reset, timeZone: c.cap.timeZone } : "not read",
resetsAt: unset ? "not set" : c.resetsAt ?? "not read",
running: child !== null,
lastExit,
snapshot: last,
@@ -493,7 +648,7 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
if (text.length > MESSAGE_MAX) throw new TriageError("triage-message-too-long", `a message holds at most ${MESSAGE_MAX} characters`);
if (queue.length >= QUEUE_MAX) throw new TriageError("triage-busy", "Triage has too many messages waiting");
const admitted = await admit();
if (!admitted.ok) throw Object.assign(new TriageError(admitted.code, admitted.code === "triage-over-cap" ? "Triage is over its daily cap" : admitted.code === "triage-no-route" ? "no model route is set for Triage" : "the model route refused"), admitted.until ? { until: admitted.until } : {});
if (!admitted.ok) throw Object.assign(new TriageError(admitted.code, admitted.code === "triage-over-cap" ? "Triage is over its daily cap" : admitted.code === "triage-no-route" ? "no model route is set for Triage" : admitted.code === "triage-usage-unreadable" ? "Triage's token usage could not be read" : admitted.code === "triage-route-unreadable" ? "the model route could not be read" : admitted.code === "triage-dir-unsafe" ? "a Triage directory is not a real directory" : "the model route refused"), admitted.until ? { until: admitted.until } : {});
await refresh();
const at = now().toISOString();
const id = `owner-${randomBytes(6).toString("hex")}`;
@@ -525,6 +680,7 @@ export function createTriageHost({ dataRoot, route = null, readSnapshot = async
}
relay?.close();
relay = null;
await usageChain;
},
};
}
+3
View File
@@ -31,6 +31,7 @@ const MESSAGES = Object.freeze({
"triage-route-refused": "the model route refused to start Triage",
"triage-route-invalid": "the model route's answer is not usable",
"triage-route-unreadable": "the model route could not be read",
"triage-usage-unreadable": "Triage's token usage could not be read, so it takes no message",
"triage-over-cap": "Triage is over its daily token cap and starts again after the time shown",
"triage-busy": "Triage has too many messages waiting",
"triage-dir-unsafe": "a Triage directory is not safe to use",
@@ -69,6 +70,8 @@ export async function triageView(triage, { send, now = Date.now } = {}) {
...(s.route === "set" ? {} : { routeMessage: NO_ROUTE_MESSAGE }),
overCapUntil: s.overCapUntil ?? null,
...(s.overCapUntil ? { capMessage: capMessage(s.overCapUntil) } : {}),
// #1873 triage-cap: today's count, the stored cap and the next reset, or "not set" / "not read".
...(Object.hasOwn(s, "usageToday") ? { usageToday: s.usageToday, cap: s.cap, resetsAt: s.resetsAt } : {}),
running: s.running,
lastExit: s.lastExit,
snapshot: s.snapshot,
+263
View File
@@ -0,0 +1,263 @@
// #1873 triage-settings: the owner's read and write of the Triage model route, for Settings > Triage.
// GET and POST /api/triage/route register through serve.mjs's route table, so the server's session
// and Origin checks run first; this module then takes the proxy-admin gate (the instance owner's own
// session on the public name, never a tester or a granted user) and the same-origin checks again.
// A write goes through the route store (triage-route-store.mjs), so the lock, the revision compare
// and the receipt are the store's. Before it, the model and harness are checked the way an
// organization's route change is (checkRoutes in packages/organization/src/launch-settings.mjs):
// offered by the catalogue, available, and compatible with the harness.
//
// The catalogue lives in each organization's bus (the trusted bus boot builds it; nothing outside
// a bus holds the proxy client), so the Console reads it through each active organization's
// `routes` verb, as the proxy setup page does, and merges the answers (mergeCatalogues).
//
// Saving a route starts nothing: the Triage host is not wired to the store in this slice.
// No key, path, env value or exception text reaches a response or a log line.
import { readdirSync, lstatSync } from 'node:fs';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { checkRoutes } from '../../organization/src/launch-settings.mjs';
import { isDailyModelQuota } from '../../plugins/src/model-quota.mjs';
import { readRoute, writeRoute } from './triage-route-store.mjs';
export const TRIAGE_ROUTE_PATH = '/api/triage/route';
export const BODY_MAX = 16384;
export const POST_KEYS = Object.freeze(['model', 'harness', 'cap', 'timeZone', 'revision']);
export const NOT_SET_MESSAGE = 'No Triage route is set.';
export const NOT_STARTED_NOTE = 'Saving a route does not start Triage. The key and the wiring are separate.';
const HARNESS_ROOT = fileURLToPath(new URL('../../plugins/harness/', import.meta.url));
// The same shapes the store holds (triage-route-store.mjs).
const MODEL_KEY = /^[a-z0-9][a-z0-9._-]{0,63}$/;
const HARNESS_NAME = /^[a-z][a-z0-9-]{0,31}$/;
const ACTOR = /^[a-z][a-z0-9-]{0,31}$/;
// The store's zone rule (its knownZone): isDailyModelQuota's zone shape, then Intl.
const knownZone = zone => typeof zone === 'string' && zone !== ''
&& isDailyModelQuota('000000000000', { limit: 0, since: '2000-01-01T00:00:00.000Z', reset: '00:00', timeZone: zone });
// The zone as Intl names it, so "america/chicago" is kept as "America/Chicago". A known zone that
// Intl renames to something the store would not take is kept as typed.
export function canonicalZone(zone) {
try {
const named = new Intl.DateTimeFormat('en-US', { timeZone: zone }).resolvedOptions().timeZone;
return knownZone(named) ? named : zone;
} catch { return zone; }
}
const STATUS = Object.freeze({
'human-required': 403,
'read-only': 403,
'cross-origin': 403,
'cross-site': 403,
'method-not-allowed': 405,
'invalid-request': 400,
'triage-cap-invalid': 400,
'triage-zone-required': 400,
'triage-zone-unknown': 400,
'unknown-model': 400,
'model-harness-incompatible': 400,
'harness-off': 400,
'triage-route-invalid': 400,
'triage-route-stale': 409,
'triage-route-busy': 409,
'model-provider-unavailable': 503,
'triage-route-unreadable': 503,
'triage-route-unrecorded': 500,
'outcome-unknown': 503,
});
// Fixed words for each code. The store's own messages name lock files, so none is forwarded.
const MESSAGES = Object.freeze({
'human-required': 'Only the instance owner can read or set the Triage route.',
'read-only': 'This Console was started without writes, so the route cannot be saved.',
'cross-origin': 'The request did not come from this Console page.',
'cross-site': 'The request did not come from this Console page.',
'method-not-allowed': 'This address takes GET and POST only.',
'invalid-request': 'The request is not a Triage route. It takes model, harness, cap, timeZone and revision.',
'triage-cap-invalid': 'The cap must be a whole number of tokens, 1 or more.',
'triage-zone-required': 'A time zone is required. There is no default.',
'triage-zone-unknown': 'That time zone is not known. Use a name like Europe/Berlin or America/Chicago.',
'unknown-model': 'The model catalogue does not offer that model.',
'model-harness-incompatible': 'That harness cannot run that model.',
'harness-off': 'No harness module of that name is installed.',
'triage-route-invalid': 'The route was refused as invalid.',
'triage-route-stale': 'The route changed since this page read it. Reload the route before saving again.',
'triage-route-busy': 'Another save of the route is in progress. Try again.',
'model-provider-unavailable': 'The model catalogue cannot confirm that model now, so the route was not saved.',
'triage-route-unreadable': 'The stored route could not be read.',
'triage-route-unrecorded': 'The route was saved, but its receipt line was not written.',
'outcome-unknown': 'The route service did not answer.',
});
const json = (res, status, value) => {
res.writeHead(status, { 'content-type': 'application/json', 'cache-control': 'no-store' });
res.end(JSON.stringify(value));
};
const one = (req, name) => (req.headersDistinct[name]?.length === 1 ? req.headersDistinct[name][0] : null);
const plain = v => v !== null && typeof v === 'object' && !Array.isArray(v);
const identity = m => JSON.stringify([m.provider, m.model, m.module, m.upstream]);
// The installed harness modules: a directory under packages/plugins/harness with an index.mjs.
export function installedHarnesses(root = HARNESS_ROOT) {
try {
return readdirSync(root).filter(name => HARNESS_NAME.test(name) && (() => {
try { return lstatSync(join(root, name, 'index.mjs')).isFile(); } catch { return false; }
})()).sort();
} catch { return []; }
}
// Each active organization's catalogue, merged into the one the Triage route is checked against.
// Only ready answers count. A key is offered when a ready catalogue lists it, and dropped when two
// catalogues disagree on what it is (provider, model, module, upstream). It is available only when
// every catalogue listing it says so, and its harnesses are those every listing names that are
// installed here. Status is ready when one catalogue is; otherwise the first status seen.
export function mergeCatalogues(list, harnesses) {
const ready = (Array.isArray(list) ? list : []).filter(c => c?.status === 'ready' && Array.isArray(c.models));
if (!ready.length) {
const first = (Array.isArray(list) ? list : []).find(c => typeof c?.status === 'string')?.status;
return { status: first && first !== 'ready' ? first : 'unavailable', models: [] };
}
const merged = new Map(), dropped = new Set();
for (const catalogue of ready) {
for (const m of catalogue.models) {
if (!plain(m) || typeof m.key !== 'string' || !MODEL_KEY.test(m.key) || dropped.has(m.key)) continue;
const names = Array.isArray(m.harnesses) ? m.harnesses.filter(h => typeof h === 'string') : [];
const held = merged.get(m.key);
if (!held) {
merged.set(m.key, { key: m.key, label: typeof m.label === 'string' && m.label ? m.label : m.key, provider: m.provider, model: m.model, module: m.module, upstream: m.upstream, available: m.available === true, harnesses: names });
continue;
}
if (identity(held) !== identity(m)) { merged.delete(m.key); dropped.add(m.key); continue; }
held.available = held.available && m.available === true;
held.harnesses = held.harnesses.filter(h => names.includes(h));
}
}
const installed = new Set(harnesses);
const models = [...merged.values()].map(m => ({ ...m, harnesses: m.harnesses.filter(h => installed.has(h)) })).sort((a, b) => a.key.localeCompare(b.key));
return { status: 'ready', models };
}
// The organization route check, called as it is for an organization's change: one role whose
// vars name the harness and model, launch.models holding the catalogue's own entry for that key
// (none when it is not offered), plugins.harness the installed harness modules, and
// plugins.model-provider the entry's module. It throws unknown-model, model-provider-unavailable,
// model-harness-incompatible or harness-off.
export function checkTriageRoute({ model, harness, catalogue, harnesses }) {
if (catalogue?.status !== 'ready') {
const error = new Error('model-provider-unavailable'); error.code = 'model-provider-unavailable'; throw error;
}
const offered = catalogue.models.find(m => m.key === model) ?? null;
const entry = offered ? { provider: offered.provider, model: offered.model, module: offered.module, upstream: offered.upstream } : null;
checkRoutes({
proposed: {
roles: { triage: { vars: { harness, model } } },
launch: { models: entry ? { [model]: entry } : {} },
plugins: { harness: [...harnesses], 'model-provider': entry ? [entry.module] : [] },
},
old: null,
levels: () => null,
catalogue,
});
}
// What the signed-in owner is recorded as: their account id, or the dev local owner's subject.
export function ownerActor(session) {
if (typeof session?.account === 'string' && ACTOR.test(session.account)) return session.account;
if (session?.provider === 'local-dev' && typeof session.sub === 'string' && ACTOR.test(session.sub)) return session.sub;
return null;
}
const view = doc => doc ? {
model: doc.model, harness: doc.harness, cap: doc.cap.limit, reset: doc.cap.reset, timeZone: doc.cap.timeZone,
revision: doc.revision, updatedAt: doc.updatedAt,
} : null;
// The panel's choices: no provider, upstream or module, only what a select shows.
const choices = catalogue => ({
status: catalogue.status,
models: catalogue.models.map(m => ({ key: m.key, label: m.label, available: m.available, harnesses: [...m.harnesses] })),
});
async function readBody(req) {
if ((req.headers['content-type'] ?? '').split(';')[0].trim().toLowerCase() !== 'application/json') throw Error();
const chunks = [];
let size = 0;
for await (const chunk of req) {
size += chunk.length;
if (size > BODY_MAX) throw Error();
chunks.push(chunk);
}
const value = JSON.parse(Buffer.concat(chunks));
if (!plain(value) || Object.keys(value).some(k => !POST_KEYS.includes(k))) throw Error();
return value;
}
// Registers both routes. `catalogues` returns each active organization's `routes` catalogue (or a
// status when its bus did not answer); `harnesses` lists the installed harness modules.
export function triageRouteRoutes({ dataRoot, catalogues, harnesses = installedHarnesses, log = () => {} }) {
async function catalogue() {
let list;
try { list = await catalogues(); } catch { list = []; }
return mergeCatalogues(list, harnesses());
}
async function handle(req, res, context) {
const done = (status, value) => {
// Method, path and status only.
log(`${new Date().toISOString()} ${req.method} ${TRIAGE_ROUTE_PATH} ${status}`);
return json(res, status, value);
};
const refuse = (code, extra = {}) => done(STATUS[code] ?? 503, { error: code, message: MESSAGES[code] ?? MESSAGES['outcome-unknown'], ...extra });
if (req.method !== 'GET' && req.method !== 'POST') return refuse('method-not-allowed');
// The proxy-admin gate: the instance owner's own session, never a tester or a granted user.
if (context.owner !== true || !context.session) return refuse('human-required');
// A write needs this Console's Origin and same-origin Fetch Metadata, exactly once each. A read
// refuses another site's Origin or Fetch Metadata and lets their absence through, as the packs and
// request reads do: a browser sends no Origin on a same-origin GET, and Chromium sends no
// Fetch Metadata to a LAN http origin (serve.mjs marks only a LAN write).
if (req.method === 'POST') {
if (one(req, 'origin') !== context.origin) return refuse('cross-origin');
if (one(req, 'sec-fetch-site') !== 'same-origin') return refuse('cross-site');
} else {
if (req.headersDistinct.origin && one(req, 'origin') !== context.origin) return refuse('cross-origin');
if (req.headersDistinct['sec-fetch-site'] && !['same-origin', 'none'].includes(one(req, 'sec-fetch-site'))) return refuse('cross-site');
}
if (req.method === 'GET') {
let doc;
try { doc = readRoute(dataRoot); } catch (error) { return refuse(error?.code === 'triage-route-unreadable' ? 'triage-route-unreadable' : 'outcome-unknown'); }
const route = view(doc);
return done(200, {
route, ...(route ? {} : { routeMessage: NOT_SET_MESSAGE }), revision: route ? route.revision : 0,
choices: choices(await catalogue()), send: context.send === true, note: NOT_STARTED_NOTE,
});
}
if (context.send !== true) return refuse('read-only');
let body;
try { body = await readBody(req); } catch { return refuse('invalid-request'); }
const { model, harness, cap, timeZone, revision } = body;
if (typeof model !== 'string' || !MODEL_KEY.test(model)) return refuse('unknown-model');
if (typeof harness !== 'string' || !HARNESS_NAME.test(harness)) return refuse('harness-off');
if (!Number.isSafeInteger(cap) || cap < 1) return refuse('triage-cap-invalid');
if (timeZone === undefined || timeZone === null || timeZone === '') return refuse('triage-zone-required');
if (!knownZone(timeZone)) return refuse('triage-zone-unknown');
const zone = canonicalZone(timeZone);
if (!Number.isSafeInteger(revision) || revision < 0) return refuse('invalid-request');
try { checkTriageRoute({ model, harness, catalogue: await catalogue(), harnesses: harnesses() }); }
catch (error) {
const code = ['unknown-model', 'model-provider-unavailable', 'model-harness-incompatible', 'harness-off'].includes(error?.code) ? error.code : 'outcome-unknown';
return refuse(code);
}
const by = ownerActor(context.session);
if (by === null) return refuse('human-required');
try {
const doc = writeRoute(dataRoot, { model, harness, limit: cap, timeZone: zone, by, revision });
return done(200, { route: view(doc), revision: doc.revision, note: NOT_STARTED_NOTE });
} catch (error) {
const code = typeof error?.code === 'string' && Object.hasOwn(MESSAGES, error.code) && error.code.startsWith('triage-route-') ? error.code : 'outcome-unknown';
if (code !== 'triage-route-stale') return refuse(code);
let stored = null;
try { stored = view(readRoute(dataRoot)); } catch { /* the conflict still stands */ }
return refuse(code, { revision: stored ? stored.revision : 0 });
}
}
return registered => {
registered.add('GET', TRIAGE_ROUTE_PATH, handle);
registered.add('POST', TRIAGE_ROUTE_PATH, handle);
};
}
+220
View File
@@ -0,0 +1,220 @@
// #1873 triage-cap: Triage's token usage and its daily cap.
//
// The Triage host appends one line per turn to <dataRoot>/triage/usage.jsonl:
// { at, run, turn, status, input, output, cacheRead, cacheWrite }
// A known turn carries four non-negative safe integers; an unknown turn carries four nulls. The file
// sits outside the run directory, so a new start never clears it. It is held to the route store's
// file checks: a regular file, one link, the running user's, mode 0600. A link, a non-file, a wrong
// mode or owner refuses the read and the append alike, and the append leaves the file as it was.
// Lines are only ever appended. Nothing here prunes the file.
//
// The cap is the daily-quota plugin's own rule (packages/plugins/limit-check/daily-quota), called
// with the organization limits shape and the stored cap as its one leaf, for one account named
// "triage". No counting rule lives here: the count, the window, the DST handling and the hit all
// come from mayLaunch and report. nextReset only asks report when the window turns over.
import { closeSync, constants, fchmodSync, fstatSync, fsyncSync, openSync, readSync, writeSync } from "node:fs";
import { join } from "node:path";
import { mayLaunch, report } from "../../plugins/limit-check/daily-quota/index.mjs";
export const USAGE_ACCOUNT = "triage";
export const USAGE_FIELDS = Object.freeze(["at", "run", "turn", "status", "input", "output", "cacheRead", "cacheWrite"]);
const KINDS = ["input", "output", "cacheWrite", "cacheRead"];
const ISO = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/;
// A run id is the host's own triage-<14 digits>; a turn id is the runner's message id, maybe with
// ":retry". Both are held to one short, plain shape so a line never carries free text.
export const ID = /^[A-Za-z0-9:._-]{1,80}$/;
// 32 MiB holds well over a hundred thousand lines; a larger file refuses rather than stalls a read.
export const USAGE_MAX_BYTES = 32 * 1024 * 1024;
const HOUR = 3600000;
export class UsageError extends Error {
constructor(why) {
super(`the Triage usage file is unreadable: ${why}`);
this.name = "UsageError";
this.code = "triage-usage-unreadable";
}
}
const unreadable = (why) => new UsageError(why);
export const usagePath = (dataRoot) => join(dataRoot, "triage", "usage.jsonl");
const plain = (v) => v !== null && typeof v === "object" && !Array.isArray(v) && Object.getPrototypeOf(v) === Object.prototype;
const canonicalTime = (s) => typeof s === "string" && ISO.test(s) && !Number.isNaN(Date.parse(s)) && new Date(s).toISOString() === s;
const count = (n) => Number.isSafeInteger(n) && n >= 0;
// One line's shape, exactly.
export function validLine(v) {
if (!plain(v) || Object.keys(v).length !== USAGE_FIELDS.length || !USAGE_FIELDS.every((k) => Object.hasOwn(v, k))) return false;
if (!canonicalTime(v.at) || typeof v.run !== "string" || !ID.test(v.run) || typeof v.turn !== "string" || !ID.test(v.turn)) return false;
if (v.status === "known") return KINDS.every((k) => count(v[k]));
if (v.status === "unknown") return KINDS.every((k) => v[k] === null);
return false;
}
// An opened file's checks, as the route store holds route.json to.
function checkFile(fd, uid) {
const st = fstatSync(fd);
if (!st.isFile()) throw unreadable("usage.jsonl is not a regular file");
if (st.nlink !== 1) throw unreadable("usage.jsonl has more than one link");
if (st.uid !== uid) throw unreadable("usage.jsonl is not owned by this user");
if ((st.mode & 0o777) !== 0o600) throw unreadable("usage.jsonl mode is not 0600");
return st;
}
const uidOf = (opts) => opts.uid ?? process.getuid();
// Append one line. A torn last line (a write that died) is closed off with a newline first, so the
// new line always stands alone and the torn one stays one bad line.
export function appendUsage(file, line, opts = {}) {
if (!validLine(line)) throw unreadable("the line does not match the usage shape");
const text = JSON.stringify(Object.fromEntries(USAGE_FIELDS.map((k) => [k, line[k]]))) + "\n";
const flags = constants.O_RDWR | constants.O_APPEND | constants.O_NOFOLLOW | constants.O_NONBLOCK;
let fd;
try { fd = openSync(file, flags); }
catch (e) {
if (e.code === "ELOOP") throw unreadable("usage.jsonl is a link");
if (e.code !== "ENOENT") throw unreadable("usage.jsonl cannot be opened");
try { fd = openSync(file, flags | constants.O_CREAT | constants.O_EXCL, 0o600); }
catch { throw unreadable("usage.jsonl cannot be made"); }
// The umask may have narrowed the mode; set it on the descriptor just made.
try { fchmodSync(fd, 0o600); } catch { closeSync(fd); throw unreadable("usage.jsonl cannot be made"); }
}
try {
const st = checkFile(fd, uidOf(opts));
let lead = "";
if (st.size > 0) {
const last = Buffer.alloc(1);
if (readSync(fd, last, 0, 1, st.size - 1) !== 1) throw unreadable("usage.jsonl cannot be read");
if (last[0] !== 0x0a) lead = "\n";
}
try {
writeSync(fd, lead + text);
fsyncSync(fd);
} catch { throw unreadable("usage.jsonl cannot be written"); }
} finally { closeSync(fd); }
}
const nulls = { input: null, output: null, cacheRead: null, cacheWrite: null };
// Every line as a daily-quota record for the triage account. A line that does not parse or does
// not match the shape (a torn last line, an edit by hand) is refused: it becomes one unknown turn,
// keyed by its line number, dated by the next good line (or the file's change time when none
// follows), and is listed in `bad` so the host can log it. No file reads as no turns.
export function readUsage(file, opts = {}) {
const maxBytes = opts.maxBytes ?? USAGE_MAX_BYTES;
let fd;
try { fd = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK); }
catch (e) {
if (e.code === "ENOENT") return { records: [], bad: [] };
if (e.code === "ELOOP") throw unreadable("usage.jsonl is a link");
throw unreadable("usage.jsonl cannot be opened");
}
let st;
let buf;
try {
st = checkFile(fd, uidOf(opts));
if (st.size > maxBytes) throw unreadable("usage.jsonl is too large");
buf = Buffer.alloc(st.size);
let got = 0;
while (got < st.size) {
const n = readSync(fd, buf, got, st.size - got, got);
if (n === 0) break;
got += n;
}
buf = buf.subarray(0, got);
} finally { closeSync(fd); }
const lines = buf.toString("utf8").split("\n");
if (lines.at(-1) === "") lines.pop();
const parsed = lines.map((text) => {
try {
const v = JSON.parse(text);
return validLine(v) ? v : null;
} catch { return null; }
});
const records = new Array(parsed.length);
const bad = [];
let nextAt = st.mtime.toISOString();
for (let i = parsed.length - 1; i >= 0; i--) {
const v = parsed[i];
if (v) {
records[i] = { account: USAGE_ACCOUNT, ...v };
nextAt = v.at;
} else {
records[i] = { account: USAGE_ACCOUNT, at: nextAt, run: null, turn: `line-${i + 1}`, status: "unknown", ...nulls };
bad.unshift({ line: i + 1 });
}
}
return { records, bad, ino: st.ino };
}
// The organization limits shape with the stored cap as its one leaf.
export const limitsFor = (cap) => ({ tokens: { daily: { [USAGE_ACCOUNT]: cap } } });
const SUBJECT = Object.freeze({ account: USAGE_ACCOUNT });
const iso = (ms) => new Date(ms).toISOString();
// The next reset after nowMs, or after the cap's `since` when that is later. A probe record at that
// instant counts while the window holding t still starts at or before it; the reset is the first
// millisecond at which report stops counting it. A day is at most 25 hours, so 49 hours always
// holds the next one.
export function nextReset(cap, nowMs) {
const from = Math.max(nowMs, Date.parse(cap.since));
const limits = limitsFor(cap);
const probe = [{ account: USAGE_ACCOUNT, at: iso(from), run: "probe", turn: "probe", status: "known", input: 1, output: 0, cacheRead: 0, cacheWrite: 0 }];
const counts = (t) => report({ limits, records: probe, now: iso(t) })[0].count > 0;
let lo = from;
let hi = from + 49 * HOUR;
if (counts(hi)) throw unreadable("the cap has no reset in the next two days");
while (hi - lo > 1) {
const mid = Math.floor((lo + hi) / 2);
if (counts(mid)) lo = mid; else hi = mid;
}
return hi;
}
// Today's count, the decision and the next reset for the stored cap.
export function capState({ cap, records, now }) {
const at = iso(now);
const limits = limitsFor(cap);
const decision = mayLaunch({ limits, subject: SUBJECT, records, now: at });
const [r] = report({ limits, records, now: at });
return {
allow: decision.allow,
hit: decision.allow ? null : decision.hit,
usageToday: { count: r.count, ...r.kinds, unknownTurns: r.unknownTurns },
resetsAt: iso(nextReset(cap, now)),
};
}
// The turn delta, copied from packages/bus/src/usage.mjs (snapshot, delta; not exported there and
// that package is not this lane's to change). Same codes, same rules: the turn's usage is the sum
// of the transcript entries that are new since the turn began; a changed or missing earlier entry
// is "transcript-rewritten", and no new entry is "no-new-usage".
const unknown = (code) => ({ status: "unknown", code });
export function snapshot(value) {
if (value?.status === "unknown") return unknown(/^transcript-(path|size|line|entries|records|changed|parse|usage|harness)$/.test(value.code) ? value.code : "source-unknown");
if (value?.status !== "known" || !value.entries || typeof value.entries !== "object" || Array.isArray(value.entries)) return unknown("source-invalid");
const entries = Object.entries(value.entries);
if (entries.length > 20000) return unknown("source-invalid");
for (const [id, v] of entries) {
if (id.length > 512 || !v || Object.keys(v).length !== 4 || KINDS.some((k) => !Number.isSafeInteger(v[k]) || v[k] < 0)) return unknown("source-invalid");
}
return { status: "known", entries: value.entries };
}
export function delta(before, after) {
if (!before || before.status !== "known") return unknown("baseline-unknown");
if (after.status !== "known") return after;
for (const [id, v] of Object.entries(before.entries)) {
if (!Object.hasOwn(after.entries, id) || KINDS.some((k) => after.entries[id][k] !== v[k])) return unknown("transcript-rewritten");
}
const sum = { input: 0, output: 0, cacheWrite: 0, cacheRead: 0 };
let added = false;
for (const [id, v] of Object.entries(after.entries)) {
if (Object.hasOwn(before.entries, id)) continue;
added = true;
for (const k of KINDS) {
sum[k] += v[k];
if (!Number.isSafeInteger(sum[k])) return unknown("usage-overflow");
}
}
return added ? { status: "known", ...sum } : unknown("no-new-usage");
}
@@ -910,3 +910,124 @@ for (const [label, answers, line] of [
assert.equal(posts.filter(p => p.url.endsWith('/agent/launch')).length, launched ? 1 : 0);
assert.equal(await b.evaluate('window.__answers.length'), 0, 'every queued answer was asked for');
});
// SP20 (#1873): an organization refusal of Stop says so in one plain line, the line Restart gives.
for (const [label, answer, line] of [
['the organization is archived', { status: 409, body: { error: 'org-archived', message: 'organization is archived' } },
/^Not stopped: Acme Studio is archived, so its agents are not changed\. Nothing changed\.$/],
['the organization is not available', { status: 404, body: { error: 'org-unavailable' } },
/^Not stopped: Acme Studio is not available to this Console\. Nothing changed\.$/],
]) test(`SP20: a Stop refusal says so in the drawer: ${label}`, async t => {
const { b, wait } = await setup(t, { hash: '#/agents?agent=acme%3Acoder' });
const d = '[data-ws-drawer="acme:coder"]';
await wait(`${open('acme:coder')} && document.querySelector('${d} [data-drawer-stop]')`);
await b.evaluate(`window.__answers.push(${JSON.stringify(answer)})`);
await b.evaluate(`document.querySelector('${d} [data-drawer-stop]').click()`);
await wait(`document.querySelector('${d} [data-drawer-stop-yes]')`);
await b.evaluate(`document.querySelector('${d} [data-drawer-stop-yes]').click()`);
await wait(`/^Not stopped: /.test(${status(d)}) && ${usable(d)}`);
assert.match(await b.evaluate(status(d)), line);
assert.equal(await b.evaluate(`document.activeElement === document.querySelector('${d} [data-drawer-stop]')`), true, 'focus is back on Stop');
assert.deepEqual((await b.evaluate('window.__posts')).map(p => [p.url, p.method]), [['/api/orgs/acme/agent/terminate', 'POST']]);
});
// SP20: each Stop keeps its own state, as each restart does. Another drawer's question or Stop does not
// lose a running Stop's line or its running state, and a Stop that ends does not drop another drawer's question.
test('SP20: two Stops in two drawers each keep their own line and running state', async t => {
const data = D => { D.acme.instances.ideas = { definition: 'ideas', running: ['run-4'] }; };
const { b, wait } = await setup(t, { hash: '#/agents?agent=acme%3Acoder', data });
const d = '[data-ws-drawer="acme:coder"]', i = '[data-ws-drawer="acme:ideas"]';
const refused = who => ({ status: 409, body: { error: 'instance-protected', message: `instance-protected: ${who} is protected from ending. Nothing changed.`, sent: false } });
const state = x => `[${status(x)}, document.querySelector('${x} [data-drawer-stop]')?.firstChild.textContent, document.querySelector('${x} [data-drawer-stop]')?.disabled, document.querySelector('${x} [data-drawer-restart]')?.disabled]`;
await wait(`${open('acme:coder')} && document.querySelector('${d} [data-drawer-stop]')`);
// Coder's Stop runs: its terminate is held.
await b.evaluate(`window.__answers.push({ hold: new Promise(r => { window.__release = r; }), ...${JSON.stringify(refused('coder'))} })`);
await b.evaluate(`document.querySelector('${d} [data-drawer-stop]').click()`);
await wait(`document.querySelector('${d} [data-drawer-stop-yes]')`);
await b.evaluate(`document.querySelector('${d} [data-drawer-stop-yes]').click()`);
await wait(`window.__posts.length === 1`);
// Ideas asks to stop and is cancelled; back on coder, its Stop still shows as running.
await b.evaluate('location.hash = "#/agents?agent=acme%3Aideas"');
await wait(`${open('acme:ideas')} && document.querySelector('${i} [data-drawer-stop]')`);
await b.evaluate(`document.querySelector('${i} [data-drawer-stop]').click()`);
await wait(`document.querySelector('${i} [data-drawer-stop-no]')`);
await b.evaluate(`document.querySelector('${i} [data-drawer-stop-no]').click()`);
await b.evaluate('location.hash = "#/agents?agent=acme%3Acoder"');
await wait(`${open('acme:coder')} && document.querySelector('${d} [data-drawer-stop]')`);
assert.deepEqual(await b.evaluate(state(d)), ['Stopping coder…', 'Stopping…', true, true]);
// Ideas' Stop question is open when coder's Stop is refused; each drawer keeps its own.
await b.evaluate('location.hash = "#/agents?agent=acme%3Aideas"');
await wait(`${open('acme:ideas')} && document.querySelector('${i} [data-drawer-stop]')`);
await b.evaluate(`document.querySelector('${i} [data-drawer-stop]').click()`);
await wait(`document.querySelector('${i} [data-drawer-stop-no]')`);
await b.evaluate('window.__release()');
await b.evaluate('new Promise(r => setTimeout(r, 300))');
await b.evaluate('location.hash = "#/agents?agent=acme%3Acoder"');
await wait(`${open('acme:coder')} && /^Not stopped: /.test(${status(d)}) && ${usable(d)}`);
assert.equal(await b.evaluate(status(d)), 'Not stopped: instance-protected: coder is protected from ending. Nothing changed.');
await b.evaluate('location.hash = "#/agents?agent=acme%3Aideas"');
await wait(`${open('acme:ideas')} && document.querySelector('${i} [data-drawer-stop]')`);
assert.equal(await b.evaluate(`!!document.querySelector('${i} [data-drawer-stop-yes]')`), true, 'ideas keeps its question');
// Ideas' Stop runs and is refused; coder's line is still there after, and ideas' is too.
await b.evaluate(`window.__answers.push(${JSON.stringify(refused('ideas'))})`);
await b.evaluate(`document.querySelector('${i} [data-drawer-stop-yes]').click()`);
await wait(`/^Not stopped: /.test(${status(i)}) && ${usable(i)}`);
await b.evaluate('location.hash = "#/agents?agent=acme%3Acoder"');
await wait(`${open('acme:coder')} && document.querySelector('${d} [data-drawer-stop]')`);
assert.deepEqual(await b.evaluate(state(d)), ['Not stopped: instance-protected: coder is protected from ending. Nothing changed.', 'Stop', false, false]);
await b.evaluate('location.hash = "#/agents?agent=acme%3Aideas"');
await wait(`${open('acme:ideas')} && document.querySelector('${i} [data-drawer-stop]')`);
assert.deepEqual(await b.evaluate(state(i)), ['Not stopped: instance-protected: ideas is protected from ending. Nothing changed.', 'Stop', false, false]);
assert.deepEqual((await b.evaluate('window.__posts')).map(p => JSON.parse(p.body).instance), ['coder', 'ideas']);
});
// SP20 review: two Stops run at once. One that ends, here the one the bus takes, which repaints the page,
// does not end the other's running state or line, and each drawer keeps its own line after.
test('SP20: two Stops running at once in two drawers each keep their own state', async t => {
const data = D => { D.acme.instances.ideas = { definition: 'ideas', running: ['run-4'] }; };
const { b, wait } = await setup(t, { hash: '#/agents?agent=acme%3Acoder', scripts: [FAST], data });
const d = '[data-ws-drawer="acme:coder"]', i = '[data-ws-drawer="acme:ideas"]';
const REQ = '0f0e0d0c-0b0a-4908-8706-050403020101';
const state = x => `[${status(x)}, document.querySelector('${x} [data-drawer-stop]')?.firstChild.textContent, document.querySelector('${x} [data-drawer-stop]')?.disabled, document.querySelector('${x} [data-drawer-restart]')?.disabled]`;
const ask = async x => {
await b.evaluate(`document.querySelector('${x} [data-drawer-stop]').click()`);
await wait(`document.querySelector('${x} [data-drawer-stop-yes]')`);
await b.evaluate(`document.querySelector('${x} [data-drawer-stop-yes]').click()`);
};
const show = async (agent, x) => {
await b.evaluate(`location.hash = "#/agents?agent=acme%3A${agent}"`);
await wait(`${open(`acme:${agent}`)} && document.querySelector('${x} [data-drawer-stop]')`);
};
// Both terminates are held: coder's the bus takes, ideas' it refuses.
await b.evaluate(`window.__answers.push(
{ hold: new Promise(r => { window.__releaseCoder = r; }), status: 200, body: { sent: true, request: '${REQ}' } },
{ hold: new Promise(r => { window.__releaseIdeas = r; }), status: 409, body: { error: 'instance-protected', message: 'instance-protected: ideas is protected from ending. Nothing changed.', sent: false } },
{ status: 200, body: { request: '${REQ}', state: 'closed' } })`);
await wait(`${open('acme:coder')} && document.querySelector('${d} [data-drawer-stop]')`);
await ask(d);
await wait('window.__posts.length === 1');
await show('ideas', i);
await ask(i);
await wait('window.__posts.length === 2');
assert.deepEqual(await b.evaluate(state(i)), ['Stopping ideas…', 'Stopping…', true, true]);
// Coder's Stop ends while ideas' drawer is on screen; ideas' Stop still runs.
await b.evaluate('window.__releaseCoder()');
await wait('window.__posts.length === 3');
await b.evaluate('new Promise(r => setTimeout(r, 300))');
await wait(`${open('acme:ideas')} && document.querySelector('${i} [data-drawer-stop]')`);
assert.deepEqual(await b.evaluate(state(i)), ['Stopping ideas…', 'Stopping…', true, true]);
await show('coder', d);
assert.deepEqual(await b.evaluate(state(d)), ['Stopped coder.', 'Stop', false, false]);
// Ideas' Stop is refused; each drawer keeps its own line.
await show('ideas', i);
assert.deepEqual(await b.evaluate(state(i)), ['Stopping ideas…', 'Stopping…', true, true]);
await b.evaluate('window.__releaseIdeas()');
await wait(`/^Not stopped: /.test(${status(i)}) && ${usable(i)}`);
assert.deepEqual(await b.evaluate(state(i)), ['Not stopped: instance-protected: ideas is protected from ending. Nothing changed.', 'Stop', false, false]);
await show('coder', d);
assert.deepEqual(await b.evaluate(state(d)), ['Stopped coder.', 'Stop', false, false]);
assert.deepEqual((await b.evaluate('window.__posts')).map(p => [p.url, p.method]), [
['/api/orgs/acme/agent/terminate', 'POST'], ['/api/orgs/acme/agent/terminate', 'POST'], [`/api/orgs/acme/agent/request?id=${REQ}`, 'GET']]);
assert.deepEqual((await b.evaluate('window.__posts')).slice(0, 2).map(p => JSON.parse(p.body).instance), ['coder', 'ideas']);
assert.equal(await b.evaluate('window.__answers.length'), 0, 'every queued answer was asked for');
});
+2 -2
View File
@@ -18,8 +18,8 @@ test('served Console browser: real board fixtures, keyboard, drafts, receipts, t
assert.equal(await b.evaluate('document.querySelector("#session-count").textContent'), '3 of 4');
await b.evaluate('document.querySelector("#hide-offline").click()');
assert.equal(await b.evaluate('document.querySelectorAll("table.sessions tbody tr").length'), 4);
// Rows 129, 154, 191, 194 and 294: chat.js, editor.js, processing.js, agents.js and federation.js are the scripts added, and #1839 adds requests.js (the intake panel under an owner's chat message), all same origin; the list is the pin, so another script is a visible change. #797 adds project-archive.js immediately after chat.js. F2 adds onboarding.js after org-create.js, and F3 adds console-restart.js after it. U2 (#1873) adds workspace-data.js and workspace.js before console-restart.js, and U4b adds tasks-board.js after workspace.js. U5 (#1873) added panes.js (Side by side) just before bus.js; SP6 replaces it with assistant.js (the Assistant page), which bus.js mounts.
assert.deepEqual(await b.evaluate('[...document.querySelectorAll("script")].map(s => s.getAttribute("src"))'), ['/editor.js', '/processing.js', '/org-create.js', '/onboarding.js', '/workspace-data.js', '/workspace.js', '/tasks-board.js', '/console-restart.js', '/display.js', '/inbox.js', '/files.js', '/project-remote.js', '/autonomy.js', '/packs.js', '/workflows.js', '/federation.js', '/brand.js', '/app.js', '/markdown-preview.js', '/projects.js', '/chat.js', '/project-archive.js', '/agents.js', '/assistant.js', '/bus.js', '/requests.js']);
// Rows 129, 154, 191, 194 and 294: chat.js, editor.js, processing.js, agents.js and federation.js are the scripts added, and #1839 adds requests.js (the intake panel under an owner's chat message), all same origin; the list is the pin, so another script is a visible change. #797 adds project-archive.js immediately after chat.js. F2 adds onboarding.js after org-create.js, and F3 adds console-restart.js after it. U2 (#1873) adds workspace-data.js and workspace.js before console-restart.js, and U4b adds tasks-board.js after workspace.js. triage-settings (#1873) adds triage-settings.js after console-restart.js. U5 (#1873) added panes.js (Side by side) just before bus.js; SP6 replaces it with assistant.js (the Assistant page), which bus.js mounts.
assert.deepEqual(await b.evaluate('[...document.querySelectorAll("script")].map(s => s.getAttribute("src"))'), ['/editor.js', '/processing.js', '/org-create.js', '/onboarding.js', '/workspace-data.js', '/workspace.js', '/tasks-board.js', '/console-restart.js', '/triage-settings.js', '/display.js', '/inbox.js', '/files.js', '/project-remote.js', '/autonomy.js', '/packs.js', '/workflows.js', '/federation.js', '/brand.js', '/app.js', '/markdown-preview.js', '/projects.js', '/chat.js', '/project-archive.js', '/agents.js', '/assistant.js', '/bus.js', '/requests.js']);
assert.equal(await b.evaluate(`document.querySelectorAll('script[src="/processing.js"]').length`), 1);
assert.equal(await b.evaluate(`document.querySelectorAll('script[src="/chat.js"]').length`), 1);
assert.equal(await b.evaluate('!!window.injected'), false);
+14 -1
View File
@@ -8,8 +8,12 @@
// SLEEP hang until killed
// REPLY-SLEEP send the person an answer (class RESULT) to this message through the tool socket, then hang until killed
// NOTE-SLEEP send the person a progress note (class INFO) on this message through the tool socket, then hang until killed
// SPEND <n> add one usage entry of <n> input tokens to the session's spend file (session/spend.jsonl,
// which the host tests' usage source reads as the transcript), then answer "spent <n>"
// GATED-SPEND <n> write work/started, wait for work/gate, then spend <n> as SPEND does
// anything else answer "echo: <body>"
import { existsSync, readFileSync, writeFileSync } from "node:fs";
import { appendFileSync, existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { randomBytes } from "node:crypto";
import { connect } from "node:net";
import { join } from "node:path";
@@ -64,6 +68,15 @@ if (body === "PROBE") {
const id = request.match(/^Message (\S+) from /m)?.[1];
await toolCall("send_message", { to: "human", body: "still looking", class: "INFO", in_reply_to: id });
setInterval(() => {}, 1000);
} else if (/^(GATED-)?SPEND \d{1,9}$/.test(body)) {
const n = Number(body.split(" ")[1]);
if (body.startsWith("GATED-")) {
writeFileSync(join(workspace, "started"), "x");
while (!existsSync(join(workspace, "gate"))) await new Promise((r) => setTimeout(r, 50));
}
mkdirSync(join(workspace, "..", "session"), { recursive: true });
appendFileSync(join(workspace, "..", "session", "spend.jsonl"), JSON.stringify({ id: randomBytes(6).toString("hex"), n }) + "\n");
process.stdout.write(`spent ${n}`);
} else if (body === "SLEEP") {
setInterval(() => {}, 1000);
} else {
@@ -63,7 +63,7 @@ function fakeHost(over = {}) {
const triage = {
snapshotAt: () => Date.now(),
refresh: async () => null,
state: async () => { h.reads += 1; return { route: h.route, running: h.running, lastExit: h.lastExit, overCapUntil: h.overCapUntil, snapshot: h.snapshot, transcript: h.transcript }; },
state: async () => { h.reads += 1; return { route: h.route, running: h.running, lastExit: h.lastExit, overCapUntil: h.overCapUntil, snapshot: h.snapshot, transcript: h.transcript, ...(Object.hasOwn(h, 'usageToday') ? { usageToday: h.usageToday, cap: h.cap, resetsAt: h.resetsAt } : {}) }; },
message: async text => {
h.sent.push(text);
const e = h.next; h.next = null;
@@ -98,6 +98,9 @@ async function setup(t, { host = fakeHost(), triage = host.triage, send = true,
// The page focuses its heading when its first render ends (bus.js). Open after that, as a person would, or
// that late focus can land after the drawer's own.
// The Settings > Triage panel script (triage-settings, #1873) is a static file every page loads, like
// every other page module; it carries no data. These checks are about Triage traffic, so they skip it.
const SETTINGS_SCRIPT = 'GET /triage-settings.js';
const open = async f => { await f.wait('document.querySelector("#triage-open")'); await f.wait('document.activeElement?.tagName === "H1"', 100).catch(() => {}); await f.click('#triage-open'); await f.wait('["off", "cap", "running", "stopped"].includes(document.querySelector("#triage-drawer #triage-state")?.dataset.kind)'); };
const type = async (f, words) => { await f.b.evaluate('document.querySelector("#triage-text").focus()'); await f.b.call('Input.insertText', { text: words }); };
const sendAndWait = async (f, before) => { await f.click('#triage-send'); await f.wait(`document.querySelector("#triage-send-status").textContent.trim() && document.querySelector("#triage-send-status").textContent !== ${JSON.stringify(before)} && !document.querySelector("#triage-send").disabled`); return f.text('#triage-send-status'); };
@@ -109,7 +112,7 @@ test('item 1: a session that is not the instance owner gets no entry, no drawer
await f.wait('document.querySelector("#dashboard-kpis")');
await sleep(1500);
assert.equal(await f.b.evaluate('document.querySelector("#triage-open, #triage-drawer, [id^=triage]")'), null);
assert.deepEqual(f.requests.filter(r => /triage/i.test(r)), []);
assert.deepEqual(f.requests.filter(r => /triage/i.test(r) && r !== SETTINGS_SCRIPT), []);
}
// A Triage host, but a signed-in session that is not the root owner.
{
@@ -119,7 +122,7 @@ test('item 1: a session that is not the instance owner gets no entry, no drawer
await sleep(1500);
assert.equal(await f.b.evaluate('document.querySelector("[id^=triage]")'), null);
assert.equal(await f.b.evaluate('document.body.innerHTML.includes("Triage")'), false, 'no Triage words in the page');
assert.deepEqual(f.requests.filter(r => /triage/i.test(r)), []);
assert.deepEqual(f.requests.filter(r => /triage/i.test(r) && r !== SETTINGS_SCRIPT), []);
}
});
@@ -135,7 +138,7 @@ test('item 1: the owner gets one Triage entry in the top bar on the dashboard an
assert.equal(await f.b.evaluate('document.querySelector("#triage-open").getAttribute("aria-expanded")'), 'true');
// From inside an organization the read is still the system route, never the organization's.
assert.ok(f.triageReads() >= 1);
assert.deepEqual(f.requests.filter(r => /triage/.test(r) && r !== 'GET /api/triage'), []);
assert.deepEqual(f.requests.filter(r => /triage/.test(r) && r !== 'GET /api/triage' && r !== SETTINGS_SCRIPT), []);
assert.ok(f.requests.includes('GET /api/config'), 'the unscoped config read');
// The same entry on the system dashboard.
await f.b.evaluate('location.hash = "#/dashboard"');
@@ -437,3 +440,33 @@ test('triage-followups: with no drawer open the first render still focuses the h
const f = await setup(t);
await f.wait('document.activeElement?.tagName === "H1"');
});
test('triage-cap: one usage line from the read: not set, the count against the cap with the reset, not read, and markup as text', async t => {
const f = await setup(t, { host: fakeHost({ usageToday: 'not set', cap: 'not set', resetsAt: 'not set' }) });
await open(f);
assert.equal(await f.text('#triage-usage'), 'Daily token cap: not set.');
assert.equal(await f.b.evaluate('document.querySelector("#triage-usage").hidden'), false);
const resetsAt = new Date(Date.now() + 5 * 3600000).toISOString();
const local = await f.b.evaluate(`window.MosaicLocalTime(${JSON.stringify(resetsAt)})`);
Object.assign(f.h, { usageToday: { count: 1234, input: 1200, output: 34, cacheWrite: 0, cacheRead: 0, unknownTurns: 2 }, cap: { limit: 200000, reset: '00:00', timeZone: 'UTC' }, resetsAt });
await f.wait('/^Today:/.test(document.querySelector("#triage-usage").textContent)', 400);
const line = await f.text('#triage-usage');
const [count, limit] = await f.b.evaluate('[(1234).toLocaleString(), (200000).toLocaleString()]');
assert.ok(line.startsWith(`Today: ${count} of ${limit} tokens. Resets at `), line);
assert.ok(line.includes(local), line);
assert.ok(line.endsWith(' 2 turns have no usage reading.'), line);
Object.assign(f.h, { usageToday: 'not read', cap: { limit: 200000, reset: '00:00', timeZone: 'UTC' }, resetsAt: 'not read' });
await f.wait('/not read/.test(document.querySelector("#triage-usage").textContent)', 400);
assert.equal(await f.text('#triage-usage'), "Today's token usage: not read.");
// A count that is not a whole number is not shown as one; markup stays text.
Object.assign(f.h, { usageToday: { count: '<img src=x onerror=alert(1)>', unknownTurns: 0 }, cap: { limit: 1.5 }, resetsAt: '<b>x</b>' });
await f.wait('/^Today: not read/.test(document.querySelector("#triage-usage").textContent)', 400);
assert.equal(await f.text('#triage-usage'), 'Today: not read of not read tokens.');
assert.equal(await f.b.evaluate('document.querySelector("#triage-usage").children.length'), 0);
});
test('triage-cap: a read without the usage fields shows no usage line', async t => {
const f = await setup(t);
await open(f);
assert.equal(await f.b.evaluate('document.querySelector("#triage-usage").hidden'), true);
});
+501 -3
View File
@@ -5,15 +5,18 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { connect } from "node:net";
import { existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
import { appendFileSync, chmodSync, existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { Client } from "../../bus/src/client.mjs";
import { REPO, RUNNER_FILES, checkHostedRoute, sandboxProblem } from "../../seat/src/sandbox.mjs";
import { isProviderVariable } from "../../seat/src/session.mjs";
import {
NOT_STARTED, NOT_TAKEN, RELAY_REFUSED, ROUTE_ENV_ALLOW, ROUTE_REFUSED, SPAWN_ENV_ALLOW, UNANSWERED, createRelay, createTriageHost, hostPrompt,
refusedHarnessName, triageEnv,
USAGE_NOT_READ, overCapNotTaken, refusedHarnessName, triageEnv,
} from "../src/triage-host.mjs";
import { writeRoute } from "../src/triage-route-store.mjs";
import { appendUsage, readUsage, usagePath } from "../src/triage-usage.mjs";
import { triageView } from "../src/triage-http.mjs";
const FIXTURE = new URL("./fixtures/triage-adapter.mjs", import.meta.url).pathname;
const skip = sandboxProblem() ?? false;
@@ -114,7 +117,8 @@ const REFUSED_TRIED = [
// The fake harness module: it declares the probe names and every refused one, so only the host's
// own refusal stops those. Its one install root is <root>/install.
const DECLARED = Object.freeze(["TRIAGE_PROBE", "FAKE_OK", "FAKE_TWO", ...REFUSED_TRIED]);
const fakeModule = (roots, names = DECLARED) => async (name) => name === "fake" ? { envNames: names, installRoots: () => roots } : null;
// Its transcript is <run>/session, where the stub adapter's SPEND writes (spendSource reads it).
const fakeModule = (roots, names = DECLARED) => async (name) => name === "fake" ? { envNames: names, installRoots: () => roots, transcript: ({ runDir }) => join(runDir, "session") } : null;
function host(t, root, opts = {}) {
const dataRoot = join(root, "data");
@@ -674,3 +678,497 @@ test("with devOwner set, a card's bus start carries the dev-owner flag", { skip,
const [reply] = await repliesOf(h, 1);
assert.deepEqual(reply.cards, [{ step: "bus-start", org: "alpha", command: "mosaic bus start alpha --dev-local-owner --detach" }]);
});
// #1873 triage-cap. The host counts each turn into <dataRoot>/triage/usage.jsonl and holds the stored
// cap through daily-quota. The usage source here reads <run>/session/spend.jsonl, which the stub
// adapter's SPEND writes (or a test writes by hand), one entry per spend.
const spendSource = {
readTurn: async ({ root }) => {
let text;
try { text = readFileSync(join(root, "spend.jsonl"), "utf8"); }
catch (e) { if (e.code === "ENOENT") return { status: "known", entries: {} }; throw e; }
const entries = {};
for (const l of text.split("\n").filter(Boolean)) {
const { id, n } = JSON.parse(l);
entries[id] = { input: n, output: 0, cacheWrite: 0, cacheRead: 0 };
}
return { status: "known", entries };
},
};
const DAY = "2026-11-18T00:00:00.000Z";
// Store a cap the way the owner's verb does: the route store, its since at `since`.
// The data root's triage directory, as the host's ensureDirs makes it (a test writes before any start).
const triageDir = (dataRoot) => {
mkdirSync(join(dataRoot, "triage"), { recursive: true, mode: 0o700 });
chmodSync(dataRoot, 0o700);
chmodSync(join(dataRoot, "triage"), 0o700);
};
const storeCap = (dataRoot, limit, timeZone = "UTC", since = DAY) => triageDir(dataRoot) ??
writeRoute(dataRoot, { model: "glm-5", harness: "claude-code", limit, timeZone, by: "owner" }, { now: () => new Date(since) });
const usageLine = (over = {}) => ({ at: "2026-11-18T10:00:00.000Z", run: "triage-20261118100000", turn: "owner-0123456789ab", status: "known", input: 10, output: 0, cacheRead: 0, cacheWrite: 0, ...over });
const usageLines = (dataRoot) => readUsage(usagePath(dataRoot)).records;
let spendId = 0;
const spend = (dataRoot, n) => {
mkdirSync(join(dataRoot, "triage", "agent", "session"), { recursive: true });
appendFileSync(join(dataRoot, "triage", "agent", "session", "spend.jsonl"), JSON.stringify({ id: `s${++spendId}`, n }) + "\n");
};
const counting = (root) => { const r = { asked: 0, admit: async () => { r.asked++; return answer(root); } }; return r; };
// A launcher that keeps the relay open and leaves the capability it was handed, so a test can make
// the runner's calls itself without the sandbox.
function standIn(root) {
const f = join(root, "stand-in");
writeFileSync(f, `#!/bin/sh\n: > '${join(root, "spawned")}'\ncat > '${join(root, "stdin.json")}'\nexec sleep 600\n`, { mode: 0o755 });
return f;
}
async function runnerClient(root, dataRoot) {
await waitFor(() => { try { return JSON.parse(readFileSync(join(root, "stdin.json"), "utf8")).cap; } catch { return null; } });
const cap = JSON.parse(readFileSync(join(root, "stdin.json"), "utf8")).cap;
return new Client({ path: join(dataRoot, "triage", "agent", "relay.sock"), cap });
}
test("cap: a hit before a start refuses with the next reset, asks no route and spawns nothing; at 00:00 in the stored zone it starts", { timeout: 30000 }, async (t) => {
const root = scratch();
let clock = Date.parse("2026-11-18T12:00:00.000Z");
const route = counting(root);
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), usageSource: spendSource, now: () => new Date(clock) });
storeCap(dataRoot, 100);
appendUsage(usagePath(dataRoot), usageLine({ input: 60, output: 30, cacheRead: 5, cacheWrite: 5 }));
await assert.rejects(h.message("hello"), (e) => e.code === "triage-over-cap" && e.until === "2026-11-19T00:00:00.000Z");
assert.equal(route.asked, 0, "the route is not asked over the cap");
assert.equal(existsSync(join(root, "spawned")), false);
assert.deepEqual((await h.state()).transcript, [], "a refused message is never written");
const s = await h.state();
assert.deepEqual([s.overCapUntil, s.resetsAt], ["2026-11-19T00:00:00.000Z", "2026-11-19T00:00:00.000Z"]);
assert.deepEqual(s.usageToday, { count: 100, input: 60, output: 30, cacheWrite: 5, cacheRead: 5, unknownTurns: 0 });
assert.deepEqual(s.cap, { limit: 100, reset: "00:00", timeZone: "UTC" });
clock = Date.parse("2026-11-18T23:59:59.999Z");
await assert.rejects(h.message("hello"), (e) => e.code === "triage-over-cap");
clock = Date.parse("2026-11-19T00:00:00.000Z");
const s2 = await h.state();
assert.deepEqual([s2.overCapUntil, s2.usageToday.count, s2.resetsAt], [null, 0, "2026-11-20T00:00:00.000Z"]);
await h.message("hello");
await waitFor(() => existsSync(join(root, "spawned")));
assert.equal(route.asked, 1);
});
test("cap: the reset is 00:00 in the stored zone on a DST day (America/Chicago, a 25-hour day)", { timeout: 30000 }, async (t) => {
const root = scratch();
let clock = Date.parse("2026-11-01T12:00:00.000Z");
const route = counting(root);
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), usageSource: spendSource, now: () => new Date(clock) });
storeCap(dataRoot, 100, "America/Chicago", "2026-10-31T00:00:00.000Z");
// 2026-11-01 in Chicago runs from 05:00Z (CDT) to 06:00Z the next day (CST).
appendUsage(usagePath(dataRoot), usageLine({ at: "2026-11-01T05:00:00.000Z", input: 100 }));
await assert.rejects(h.message("a"), (e) => e.code === "triage-over-cap" && e.until === "2026-11-02T06:00:00.000Z");
clock = Date.parse("2026-11-02T05:59:59.999Z");
await assert.rejects(h.message("b"), (e) => e.code === "triage-over-cap" && e.until === "2026-11-02T06:00:00.000Z");
clock = Date.parse("2026-11-02T06:00:00.000Z");
await h.message("c");
await waitFor(() => existsSync(join(root, "spawned")));
assert.equal(route.asked, 1);
// A line from before the day began does not count: 04:59:59.999Z was still 2026-10-31 in Chicago.
const other = scratch();
const o = host(t, other, { route: counting(other), bwrap: spy(other), usageSource: spendSource, now: () => new Date("2026-11-01T12:00:00.000Z") });
storeCap(o.dataRoot, 100, "America/Chicago", "2026-10-30T00:00:00.000Z");
appendUsage(usagePath(o.dataRoot), usageLine({ at: "2026-11-01T04:59:59.999Z", input: 100 }));
assert.equal((await o.h.state()).usageToday.count, 0);
});
test("cap: the relay writes one line per turn, known or unknown, and a session that ends mid-turn still gets its line", { timeout: 30000 }, async (t) => {
const root = scratch();
const logs = [];
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
const first = await h.message("one");
const c = await runnerClient(root, dataRoot);
const run = JSON.parse(readFileSync(join(dataRoot, "triage", "agent", "session.json"), "utf8")).run;
assert.match(run, /^triage-\d{14}$/);
spend(dataRoot, 7); // before the turn: the baseline holds it
assert.deepEqual(await c.call("usage.begin", { turn: first.id }), {});
spend(dataRoot, 40);
// A repeated begin for the same turn keeps the first baseline, so the tokens before it still count.
assert.deepEqual(await c.call("usage.begin", { turn: first.id }), {});
spend(dataRoot, 2);
await c.call("usage.end", { turn: first.id });
// A repeated end does not count the turn twice.
await c.call("usage.end", { turn: first.id });
await c.call("usage.begin", { turn: `${first.id}:retry` });
await c.call("usage.end", { turn: `${first.id}:retry` }); // nothing new: unknown
const lines = usageLines(dataRoot);
assert.deepEqual(lines.map((l) => [l.run, l.turn, l.status, l.input, l.output, l.cacheRead, l.cacheWrite]), [
[run, first.id, "known", 42, 0, 0, 0],
[run, `${first.id}:retry`, "unknown", null, null, null, null],
]);
assert.ok(lines.every((l) => l.at === "2026-11-18T12:00:00.000Z"), "the line's time is the host's clock");
assert.ok(logs.includes("triage host: usage.end repeated for a turn; not counted twice"));
assert.ok(logs.includes("triage host: turn usage unknown: no-new-usage"));
assert.equal(lstatSync(usagePath(dataRoot)).mode & 0o777, 0o600);
// A turn begun and never ended: the session is killed; the host writes its line.
await c.call("usage.begin", { turn: "owner-0123456789ff" });
spend(dataRoot, 5);
process.kill(-h.pid, "SIGKILL");
await waitFor(() => !h.running);
await h.stop();
assert.deepEqual(usageLines(dataRoot).slice(2).map((l) => [l.turn, l.status, l.input]), [["owner-0123456789ff", "known", 5]]);
assert.equal(usageLines(dataRoot).length, 3, "the killed turn has one line, not two");
});
test("cap: a turn that crosses the cap finishes; the runner is then refused and every queued message gets a host line", { timeout: 30000 }, async (t) => {
const root = scratch();
const logs = [];
const route = counting(root);
const { h, dataRoot } = host(t, root, { route, bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
storeCap(dataRoot, 100);
const first = await h.message("one");
const c = await runnerClient(root, dataRoot);
assert.deepEqual((await c.call("message.receive")).map((m) => m.id), [first.id]);
await c.call("state");
await c.call("usage.begin", { turn: first.id });
// Queued while the turn runs, under the cap: admitted.
const second = await h.message("two");
const third = await h.message("three");
spend(dataRoot, 150);
// Inside the turn (a continuation's state read) the runner is not stopped.
assert.equal((await c.call("state")).tasks.total, 0);
await c.call("usage.end", { turn: first.id });
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: first.id });
await c.call("message.read", { id: first.id });
// The next receive is refused: the session is gone, the queue drains with a host line each.
await assert.rejects(c.call("message.receive"), (e) => e.code === "not-holder");
const tr = (await h.state()).transcript;
assert.deepEqual(tr.filter((e) => e.from === "triage").map((e) => [e.in_reply_to, e.text]), [[first.id, "answer"]]);
const until = "2026-11-19T00:00:00.000Z";
assert.deepEqual(tr.filter((e) => e.from === "host").map((e) => [e.in_reply_to, e.text]), [[second.id, overCapNotTaken(until)], [third.id, overCapNotTaken(until)]]);
assert.ok(logs.includes("triage host: stopping the session at message.receive: over the daily token cap"), logs.join(" | "));
await assert.rejects(h.message("four"), (e) => e.code === "triage-over-cap" && e.until === until);
assert.equal(route.asked, 3);
// The session ends: nothing is left to answer, so no second line.
process.kill(-h.pid, "SIGKILL");
await waitFor(() => !h.running);
assert.equal((await h.state()).transcript.filter((e) => e.from === "host").length, 2);
});
test("cap: messages taken together: once a turn crosses the cap, the next message's turn does not start", { timeout: 30000 }, async (t) => {
const root = scratch();
const logs = [];
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
storeCap(dataRoot, 100);
const [a, b] = await Promise.all([h.message("one"), h.message("two")]);
const c = await runnerClient(root, dataRoot);
assert.deepEqual((await c.call("message.receive")).map((m) => m.id), [a.id, b.id]);
await c.call("state");
await c.call("usage.begin", { turn: a.id });
spend(dataRoot, 100);
await c.call("usage.end", { turn: a.id });
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: a.id });
await c.call("message.read", { id: a.id });
// The runner reads the state before each message's turn: refused, so b's turn never begins.
await assert.rejects(c.call("state"), (e) => e.code === "not-holder");
const hostLines = (await h.state()).transcript.filter((e) => e.from === "host");
assert.deepEqual(hostLines.map((e) => [e.in_reply_to, e.text]), [[b.id, overCapNotTaken("2026-11-19T00:00:00.000Z")]]);
assert.ok(logs.includes("triage host: stopping the session at state: over the daily token cap"), logs.join(" | "));
assert.equal(usageLines(dataRoot).length, 1);
});
test("cap: a restart counts what the file holds: a new host on the same data root refuses at the same count", { timeout: 30000 }, async (t) => {
const root = scratch();
const first = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
storeCap(first.dataRoot, 100);
const m = await first.h.message("one");
const c = await runnerClient(root, first.dataRoot);
await c.call("usage.begin", { turn: m.id });
spend(first.dataRoot, 60);
await c.call("usage.end", { turn: m.id });
// The second turn is open when the host stops: its line is written on the way down.
await c.call("usage.begin", { turn: "owner-0123456789ee" });
spend(first.dataRoot, 45);
await first.h.stop();
assert.deepEqual(usageLines(first.dataRoot).map((l) => [l.turn, l.input]), [[m.id, 60], ["owner-0123456789ee", 45]]);
const route = counting(root);
const second = host(t, root, { route, bwrap: spy(root), usageSource: spendSource, now: () => new Date("2026-11-18T13:00:00.000Z") });
assert.equal((await second.h.state()).usageToday.count, 105);
await assert.rejects(second.h.message("two"), (e) => e.code === "triage-over-cap" && e.until === "2026-11-19T00:00:00.000Z");
assert.equal(route.asked, 0);
});
test("cap: a message whose turn began and was not answered gets the unanswered line when the cap stops the session", { timeout: 30000 }, async (t) => {
const root = scratch();
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
storeCap(dataRoot, 100);
const [a, b] = await Promise.all([h.message("one"), h.message("two")]);
const c = await runnerClient(root, dataRoot);
await c.call("message.receive");
await c.call("usage.begin", { turn: a.id });
spend(dataRoot, 100);
await c.call("usage.end", { turn: a.id });
// The turn's answer is lost (a halted processing state leaves a message unread): a stays queued.
await assert.rejects(c.call("state"), (e) => e.code === "not-holder");
const lines = (await h.state()).transcript.filter((e) => e.from === "host").map((e) => [e.in_reply_to, e.text]);
assert.deepEqual(lines, [[a.id, UNANSWERED], [b.id, overCapNotTaken("2026-11-19T00:00:00.000Z")]]);
});
test("cap: under the sandbox, the runner's turns are counted, a turn that crosses the cap answers, the queued message drains, and a restart counts the file", { skip, timeout: 120000 }, async (t) => {
const root = scratch();
const logs = [];
const route = stubRoute(root);
const clock = () => new Date("2026-11-18T12:00:00.000Z");
const { h, dataRoot } = host(t, root, { route, usageSource: spendSource, now: clock, log: (l) => logs.push(l) });
storeCap(dataRoot, 1000);
const one = await h.message("SPEND 300");
const echo = await h.message("plain");
const replies = await repliesOf(h, 2);
assert.deepEqual(replies.map((e) => [e.text.replace(/\n\n.*$/s, ""), e.in_reply_to]), [["spent 300", one.id], ["echo: plain", echo.id]]);
const lines = usageLines(dataRoot);
assert.deepEqual(lines.map((l) => [l.turn, l.status, l.input]), [[one.id, "known", 300], [echo.id, "unknown", null]]);
assert.match(lines[0].run, /^triage-\d{14}$/);
await h.stop();
// A restart: a new host on the same data root counts the file.
const again = host(t, root, { route, usageSource: spendSource, now: clock, log: (l) => logs.push(l) });
const s = await again.h.state();
assert.deepEqual([s.usageToday.count, s.usageToday.unknownTurns], [300, 1]);
const big = await again.h.message("GATED-SPEND 800");
await waitFor(() => existsSync(join(dataRoot, "triage", "agent", "work", "started")));
const queued = await again.h.message("queued");
writeFileSync(join(dataRoot, "triage", "agent", "work", "gate"), "x");
const [answer] = await repliesOf(again.h, 3).then((l) => l.slice(2));
assert.deepEqual([answer.text.replace(/\n\n.*$/s, ""), answer.in_reply_to], ["spent 800", big.id]);
const [note] = await repliesOf(again.h, 1, "host");
assert.deepEqual([note.in_reply_to, note.text], [queued.id, overCapNotTaken("2026-11-19T00:00:00.000Z")]);
await waitFor(() => !again.h.running);
assert.equal((await again.h.state()).lastExit.code, 22, "the runner ends as a session that is gone");
assert.equal((await again.h.state()).transcript.filter((e) => e.from === "host").length, 1);
await assert.rejects(again.h.message("after"), (e) => e.code === "triage-over-cap");
assert.equal(usageLines(dataRoot).at(-1).input, 800);
});
test("cap: a hit from another count before a queued message: the file is read again at each check", { timeout: 30000 }, async (t) => {
const root = scratch();
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
storeCap(dataRoot, 1000);
const first = await h.message("one");
const c = await runnerClient(root, dataRoot);
await c.call("message.receive");
const second = await h.message("two");
// Under the cap the runner takes the queued message.
assert.deepEqual((await c.call("message.receive")).map((m) => m.id), [second.id]);
await c.call("usage.begin", { turn: second.id });
const third = await h.message("three");
appendUsage(usagePath(dataRoot), usageLine({ input: 1000 }));
// Over the cap inside second's turn: a continuation's state read is not stopped; the turn finishes.
assert.equal((await c.call("state")).tasks.total, 0);
spend(dataRoot, 4);
await c.call("usage.end", { turn: second.id });
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: second.id });
await c.call("message.read", { id: second.id });
await assert.rejects(c.call("message.receive"), (e) => e.code === "not-holder");
const until = overCapNotTaken("2026-11-19T00:00:00.000Z");
const hostLines = (await h.state()).transcript.filter((e) => e.from === "host").map((e) => [e.in_reply_to, e.text]);
assert.deepEqual(hostLines.sort(), [[first.id, until], [third.id, until]].sort(), "every message still queued leaves it");
assert.deepEqual(usageLines(dataRoot).at(-1).input, 4, "the finished turn is counted");
});
test("cap: an idle session over the cap ends at its next receive, with nothing queued", { timeout: 30000 }, async (t) => {
const root = scratch();
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
storeCap(dataRoot, 1000);
const m = await h.message("one");
const c = await runnerClient(root, dataRoot);
await c.call("message.receive");
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: m.id });
await c.call("message.read", { id: m.id });
appendUsage(usagePath(dataRoot), usageLine({ input: 1000 }));
await assert.rejects(c.call("message.receive"), (e) => e.code === "not-holder");
assert.equal((await h.state()).transcript.filter((e) => e.from === "host").length, 0, "nothing was queued, so no host line");
});
test("cap: the count waits for a turn still being counted: usage.end answers after its line, and a message right after a mid-turn kill is refused", { timeout: 30000 }, async (t) => {
const root = scratch();
let gate = null; // while set, the usage source waits on it
const close = () => { let release; const promise = new Promise((r) => { release = r; }); gate = { promise, release: () => { gate = null; release(); } }; return gate; };
const gated = { readTurn: async (a) => { if (gate) await gate.promise; return spendSource.readTurn(a); } };
const route = counting(root);
// A failed assertion must not leave the gate shut: the host's stop waits for the usage chain. After hooks
// run in the order they were added, so this one goes before the host's stop.
t.after(() => gate?.release());
const { h, dataRoot } = host(t, root, { route, bwrap: standIn(root), usageSource: gated, now: () => new Date("2026-11-18T12:00:00.000Z") });
storeCap(dataRoot, 100);
const m = await h.message("one");
const c = await runnerClient(root, dataRoot);
await c.call("message.receive");
await c.call("usage.begin", { turn: m.id });
spend(dataRoot, 60);
let g = close();
let answered = false;
const end = c.call("usage.end", { turn: m.id }).then(() => { answered = true; });
await new Promise((r) => setTimeout(r, 200));
assert.equal(answered, false, "usage.end is not answered before its line is written");
assert.equal(usageLines(dataRoot).length, 0);
g.release();
await end;
assert.deepEqual(usageLines(dataRoot).map((l) => l.input), [60]);
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: m.id });
await c.call("message.read", { id: m.id });
// A second turn crosses the cap and the session is killed before its usage.end.
const two = await h.message("two");
assert.deepEqual((await c.call("message.receive")).map((x) => x.id), [two.id]);
await c.call("usage.begin", { turn: two.id });
spend(dataRoot, 50);
g = close();
process.kill(-h.pid, "SIGKILL");
await waitFor(() => !h.running);
const asked = route.asked;
setTimeout(() => g.release(), 200);
await assert.rejects(h.message("three"), (e) => e.code === "triage-over-cap" && e.until === "2026-11-19T00:00:00.000Z");
assert.equal(route.asked, asked, "the route is not asked: the open turn was counted first");
assert.deepEqual(usageLines(dataRoot).map((l) => [l.turn, l.input]), [[m.id, 60], [two.id, 50]]);
});
test("cap: a harness transcript path outside the run directory reads as no transcript, so the turn's line is unknown", { timeout: 30000 }, async (t) => {
for (const label of ["a sibling of the run directory", "a path through .. from inside it"]) {
const root = scratch();
const logs = [];
let target = null;
const mod = async (name) => name === "fake" ? { envNames: DECLARED, installRoots: () => [join(root, "install")], transcript: ({ runDir }) => (target = label.startsWith("a sibling") ? join(runDir, "..", "outside") : `${runDir}/session/../../outside`) } : null;
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, harnessModule: mod, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
const m = await h.message("one");
const c = await runnerClient(root, dataRoot);
mkdirSync(target, { recursive: true });
await c.call("usage.begin", { turn: m.id });
appendFileSync(join(target, "spend.jsonl"), JSON.stringify({ id: "x1", n: 5 }) + "\n");
await c.call("usage.end", { turn: m.id });
assert.deepEqual(usageLines(dataRoot).map((l) => [l.status, l.input]), [["unknown", null]], label);
assert.ok(logs.includes("triage host: turn usage unknown: baseline-unknown"), label);
}
});
test("cap: a torn last line is refused, logged once, and counts as an unknown turn; the next line stands alone", { timeout: 30000 }, async (t) => {
const root = scratch();
const logs = [];
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
storeCap(dataRoot, 1000);
appendUsage(usagePath(dataRoot), usageLine({ input: 30 }));
appendFileSync(usagePath(dataRoot), '{"at":"2026-11-18T11:00:00.000Z","run":"triage-2026111');
const s = await h.state();
assert.deepEqual([s.usageToday.count, s.usageToday.unknownTurns], [30, 1]);
await h.state();
assert.deepEqual(logs.filter((l) => /usage\.jsonl line/.test(l)), ["triage host: usage.jsonl line 2 is not a usage line; it counts as an unknown turn"]);
// Under the cap, a message starts, and its turn's line is written after the torn one.
const m = await h.message("one");
const c = await runnerClient(root, dataRoot);
await c.call("usage.begin", { turn: m.id });
spend(dataRoot, 20);
await c.call("usage.end", { turn: m.id });
const lines = usageLines(dataRoot);
assert.deepEqual(lines.map((l) => [l.turn, l.status, l.input]), [["owner-0123456789ab", "known", 30], ["line-2", "unknown", null], [m.id, "known", 20]]);
const s2 = await h.state();
assert.deepEqual([s2.usageToday.count, s2.usageToday.unknownTurns], [50, 1]);
});
test("cap: a usage file with a wrong mode, a link or a non-file refuses every message and reads as not read", { timeout: 30000 }, async (t) => {
const cases = [
["mode 0644", (f) => { appendUsage(f, usageLine()); chmodSync(f, 0o644); }],
["mode 0400", (f) => { appendUsage(f, usageLine()); chmodSync(f, 0o400); }],
["a link", (f) => { writeFileSync(f + ".real", JSON.stringify(usageLine()) + "\n", { mode: 0o600 }); symlinkSync(f + ".real", f); }],
["a directory", (f) => mkdirSync(f)],
];
for (const [label, make] of cases) {
const root = scratch();
const route = counting(root);
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
storeCap(dataRoot, 1000);
make(usagePath(dataRoot));
await assert.rejects(h.message("hello"), (e) => e.code === "triage-usage-unreadable" && e.message === "Triage's token usage could not be read", label);
assert.equal(route.asked, 0, label);
assert.equal(existsSync(join(root, "spawned")), false, label);
const s = await h.state();
assert.deepEqual([s.usageToday, s.cap, s.resetsAt], ["not read", { limit: 1000, reset: "00:00", timeZone: "UTC" }, "2026-11-19T00:00:00.000Z"], label);
}
});
test("cap: a line the host cannot write leaves the count short, so the stored cap refuses from then on", { timeout: 30000 }, async (t) => {
const root = scratch();
const logs = [];
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z"), log: (l) => logs.push(l) });
storeCap(dataRoot, 1000);
const m = await h.message("one");
const c = await runnerClient(root, dataRoot);
await c.call("message.receive");
await c.call("usage.begin", { turn: m.id });
spend(dataRoot, 5);
appendUsage(usagePath(dataRoot), usageLine({ input: 1 }));
chmodSync(usagePath(dataRoot), 0o644);
await c.call("usage.end", { turn: m.id });
assert.ok(logs.includes("triage host: usage line not written: triage-usage-unreadable"), logs.join(" | "));
chmodSync(usagePath(dataRoot), 0o600);
assert.equal(usageLines(dataRoot).length, 1, "the file is as it was");
// The file reads again, but a turn is missing from it: refused, and the state says not read.
await assert.rejects(h.message("two"), (e) => e.code === "triage-usage-unreadable");
assert.equal((await h.state()).usageToday, "not read");
await c.call("message.send", { to: "human", body: "answer", class: "RESULT", in_reply_to: m.id });
const later = await h.state();
assert.equal(later.transcript.filter((e) => e.from === "host").length, 0);
});
test("cap: with no route stored the fields say not set, nothing is refused for usage, and each turn's line is still written", { timeout: 30000 }, async (t) => {
const root = scratch();
const { h, dataRoot } = host(t, root, { route: counting(root), bwrap: standIn(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
const s = await h.state();
assert.deepEqual([s.usageToday, s.cap, s.resetsAt, s.overCapUntil], ["not set", "not set", "not set", null]);
triageDir(dataRoot);
// A usage file over any count, even a broken one, refuses nothing while no cap is stored.
appendUsage(usagePath(dataRoot), usageLine({ input: 10 ** 9 }));
appendFileSync(usagePath(dataRoot), "not json");
const m = await h.message("one");
const c = await runnerClient(root, dataRoot);
assert.deepEqual((await c.call("message.receive")).map((x) => x.id), [m.id]);
await c.call("state");
await c.call("usage.begin", { turn: m.id });
spend(dataRoot, 3);
await c.call("usage.end", { turn: m.id });
assert.deepEqual(usageLines(dataRoot).at(-1).input, 3);
const two = await h.message("two");
assert.deepEqual((await c.call("message.receive")).map((x) => x.id), [two.id]);
});
test("cap: an unreadable route store refuses every message and reads as not read", { timeout: 30000 }, async (t) => {
const root = scratch();
const route = counting(root);
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
storeCap(dataRoot, 1000);
chmodSync(join(dataRoot, "triage", "route.json"), 0o644);
await assert.rejects(h.message("hello"), (e) => e.code === "triage-route-unreadable" && e.message === "the model route could not be read");
assert.equal(route.asked, 0);
const s = await h.state();
assert.deepEqual([s.usageToday, s.cap, s.resetsAt], ["not read", "not read", "not read"]);
// A running session: the store turns unreadable, the next receive is refused and the queue drains.
const other = scratch();
const r = host(t, other, { route: counting(other), bwrap: standIn(other), usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
storeCap(r.dataRoot, 1000);
const m = await r.h.message("one");
const c = await runnerClient(other, r.dataRoot);
await c.call("message.receive");
const two = await r.h.message("two");
chmodSync(join(r.dataRoot, "triage", "route.json"), 0o644);
await assert.rejects(c.call("message.receive"), (e) => e.code === "not-holder");
const hostLines = (await r.h.state()).transcript.filter((e) => e.from === "host").map((e) => [e.in_reply_to, e.text]);
assert.deepEqual(hostLines.sort(), [[m.id, USAGE_NOT_READ], [two.id, USAGE_NOT_READ]].sort());
});
test("cap: the read carries usageToday, cap and resetsAt, with and without a stored route, and no path, key or env value", { timeout: 30000 }, async (t) => {
const root = scratch();
const env = { ...CONSOLE_ENV, ANTHROPIC_AUTH_TOKEN: "tok-secret-value" };
const route = { admit: async () => answer(root, { env: { ANTHROPIC_BASE_URL: "http://127.0.0.1:9", ANTHROPIC_AUTH_TOKEN: "tok-secret-value" } }) };
const { h, dataRoot } = host(t, root, { route, bwrap: spy(root), env, usageSource: spendSource, now: () => new Date("2026-11-18T12:00:00.000Z") });
const unset = await triageView(h, { send: true });
assert.deepEqual([unset.usageToday, unset.cap, unset.resetsAt], ["not set", "not set", "not set"]);
storeCap(dataRoot, 500, "Asia/Kolkata", "2026-11-17T00:00:00.000Z");
appendUsage(usagePath(dataRoot), usageLine({ at: "2026-11-18T11:00:00.000Z", input: 120, output: 3 }));
await h.message("hello");
const view = await triageView(h, { send: true });
assert.deepEqual(view.usageToday, { count: 123, input: 120, output: 3, cacheWrite: 0, cacheRead: 0, unknownTurns: 0 });
assert.deepEqual(view.cap, { limit: 500, reset: "00:00", timeZone: "Asia/Kolkata" });
assert.equal(view.resetsAt, "2026-11-18T18:30:00.000Z");
const text = JSON.stringify([unset, view]);
for (const bad of [dataRoot, root, "usage.jsonl", "route.json", "glm-5", "claude-code", "tok-secret-value", "from-the-console", "since", "2026-11-17T00:00:00.000Z", "triage-2026"]) {
assert.equal(text.includes(bad), false, bad);
}
});
+18 -1
View File
@@ -4,7 +4,7 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { request } from 'node:http';
import { mkdtempSync, existsSync, readFileSync } from 'node:fs';
import { mkdtempSync, mkdirSync, existsSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import { startServer } from '../src/serve.mjs';
import { createDevOwner } from '../src/dev-owner.mjs';
@@ -12,6 +12,8 @@ import { consoleAccess } from '../src/access.mjs';
import { createSsoSession } from './helpers/sso-session.mjs';
import { NO_ROUTE_MESSAGE, capMessage, triageView, triageHTTP, SNAPSHOT_MAX_AGE_MS } from '../src/triage-http.mjs';
import { createTriageHost, TriageError } from '../src/triage-host.mjs';
import { writeRoute } from '../src/triage-route-store.mjs';
import { appendUsage, usagePath } from '../src/triage-usage.mjs';
const LEAK = '/home/owner/.config/mosaic-dev/secret-token-0123456789';
@@ -239,6 +241,21 @@ test('with the real host and no route: the read says so in plain words, the writ
assert.equal(JSON.parse(readFileSync(join(host.paths.health, 'snapshot.json'), 'utf8')).snapshotVersion, 1, 'the read wrote the snapshot');
});
test('triage-cap: the read carries usageToday, cap and resetsAt from the real host, and no path, model or harness', async t => {
const dataRoot = join(mkdtempSync(join(process.env.TMPDIR, 'th-')), 'data');
mkdirSync(join(dataRoot, 'triage'), { recursive: true, mode: 0o700 });
writeRoute(dataRoot, { model: 'glm-5', harness: 'claude-code', limit: 1000, timeZone: 'UTC', by: 'owner' }, { now: () => new Date('2026-11-18T00:00:00.000Z') });
appendUsage(usagePath(dataRoot), { at: '2026-11-18T09:00:00.000Z', run: 'triage-20261118090000', turn: 'owner-0123456789ab', status: 'known', input: 30, output: 5, cacheRead: 2, cacheWrite: 1 });
const host = createTriageHost({ dataRoot, readSnapshot: async () => null, now: () => new Date('2026-11-18T10:00:00.000Z'), bwrap: '/nonexistent/bwrap' });
const f = await devServer(t, { triage: host });
const read = await send(f.port, '/api/triage', { headers: { host: f.headers.host } });
assert.equal(read.status, 200);
assert.deepEqual(read.json.usageToday, { count: 38, input: 30, output: 5, cacheWrite: 1, cacheRead: 2, unknownTurns: 0 });
assert.deepEqual(read.json.cap, { limit: 1000, reset: '00:00', timeZone: 'UTC' });
assert.equal(read.json.resetsAt, '2026-11-19T00:00:00.000Z');
for (const leak of [dataRoot, 'glm-5', 'claude-code', 'usage.jsonl', '"since"']) assert.equal(read.text.includes(leak), false, leak);
});
test('the route module keeps its own write checks when called alone: --send, one exact Origin, one same-origin Fetch Metadata', async () => {
const triage = fakeTriage();
const call = async (headers, { send = true } = {}) => {
@@ -0,0 +1,393 @@
// #1873 triage-settings: GET and POST /api/triage/route over the real HTTP boundary. Only the instance
// owner reaches them, a write is same-origin, the model and harness are checked as an organization's
// route change is, the store does the write (one receipt), and no key or path reaches a response.
import test from 'node:test';
import assert from 'node:assert/strict';
import { request } from 'node:http';
import { mkdtempSync, readFileSync, rmSync, writeFileSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { startServer } from '../src/serve.mjs';
import { createDevOwner } from '../src/dev-owner.mjs';
import { consoleAccess } from '../src/access.mjs';
import { createSsoSession } from './helpers/sso-session.mjs';
import { routePaths, readRoute, writeRoute } from '../src/triage-route-store.mjs';
import { triageRouteRoutes, mergeCatalogues, installedHarnesses, ownerActor, TRIAGE_ROUTE_PATH, NOT_STARTED_NOTE } from '../src/triage-route-http.mjs';
const SECRET = 'sk-triage-route-sentinel-0123456789abcdef';
const T = '2026-11-18T12:00:00.000Z';
const model = (key, extra = {}) => ({ key, module: 'anthropic', upstream: 'cliproxy', provider: 'anthropic', model: `claude-${key}`, label: key.toUpperCase(), available: true, reason: null, harnesses: ['claude-code', 'pi'], ...extra });
// One catalogue as an organization's `routes` verb answers. A leaked field would carry the sentinel.
const READY = () => ({ status: 'ready', observedAt: T, notices: [`key ${SECRET} at /srv/proxy/keys.json`], models: [
model('sonnet', { reason: `${SECRET} /home/owner/.config/mosaic-dev/proxy.json` }),
model('glm-flash', { module: 'opencode', provider: 'zai', model: 'glm-5.3-flash', harnesses: ['pi'] }),
model('down', { available: false, reason: 'proxy says no' }),
] });
const BODY = { model: 'sonnet', harness: 'claude-code', cap: 200000, timeZone: 'Europe/Berlin', revision: 0 };
const send = (port, path, { method = 'GET', headers = {}, body } = {}) => new Promise((resolve, reject) => {
const req = request({ host: '127.0.0.1', port, path, method, headers }, res => {
let text = ''; res.on('data', c => { text += c; }); res.on('end', () => resolve({ status: res.statusCode, text, json: (() => { try { return JSON.parse(text); } catch { return null; } })() }));
});
req.on('error', reject); req.end(body === undefined ? undefined : typeof body === 'string' ? body : JSON.stringify(body));
});
const receipts = root => { const f = routePaths(root).receipts; return existsSync(f) ? readFileSync(f, 'utf8').split('\n').filter(Boolean).map(l => JSON.parse(l)) : []; };
async function devServer(t, { catalogues = async () => [READY()], harnesses = () => ['claude-code', 'pi'], ...options } = {}) {
const root = mkdtempSync(join(tmpdir(), 'tr-http-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
const logs = [], routeLogs = [];
const s = await startServer({ port: 0, board: 'http://127.0.0.1:9', sso: createDevOwner(), send: async () => ({}), log: line => logs.push(line),
routes: [triageRouteRoutes({ dataRoot: root, catalogues, harnesses, log: line => routeLogs.push(line) })], ...options });
t.after(() => new Promise(r => s.close(r)));
const port = s.address().port;
const origin = `http://127.0.0.1:${port}`;
const read = { host: `127.0.0.1:${port}`, 'sec-fetch-site': 'same-origin' };
const write = { ...read, origin, 'content-type': 'application/json' };
const get = (headers = read) => send(port, TRIAGE_ROUTE_PATH, { headers });
const post = (body = BODY, headers = write) => send(port, TRIAGE_ROUTE_PATH, { method: 'POST', headers, body });
return { s, port, root, logs, routeLogs, read, write, get, post };
}
test('GET without a stored route says it is not set and offers the catalogue models with their harnesses', async t => {
const f = await devServer(t);
const r = await f.get();
assert.equal(r.status, 200);
assert.equal(r.json.route, null);
assert.equal(r.json.routeMessage, 'No Triage route is set.');
assert.equal(r.json.revision, 0);
assert.equal(r.json.send, true);
assert.equal(r.json.note, NOT_STARTED_NOTE);
assert.deepEqual(r.json.choices, { status: 'ready', models: [
{ key: 'down', label: 'DOWN', available: false, harnesses: ['claude-code', 'pi'] },
{ key: 'glm-flash', label: 'GLM-FLASH', available: true, harnesses: ['pi'] },
{ key: 'sonnet', label: 'SONNET', available: true, harnesses: ['claude-code', 'pi'] },
] });
assert.equal(r.text.includes(SECRET), false);
assert.equal(r.text.includes(f.root), false);
assert.equal(r.text.includes('/srv/') || r.text.includes('/home/'), false);
});
test('GET with a stored route returns its model, harness, cap, reset, zone, revision and time, and no path or actor detail', async t => {
const f = await devServer(t);
writeRoute(f.root, { model: 'sonnet', harness: 'pi', limit: 5000, timeZone: 'America/Chicago', by: 'local-dev-owner' }, { now: () => new Date(T) });
const r = await f.get();
assert.equal(r.status, 200);
assert.deepEqual(r.json.route, { model: 'sonnet', harness: 'pi', cap: 5000, reset: '00:00', timeZone: 'America/Chicago', revision: 1, updatedAt: T });
assert.equal(r.json.revision, 1);
assert.equal(Object.hasOwn(r.json, 'routeMessage'), false);
assert.equal(r.text.includes(f.root), false);
});
test('POST writes through the store: one receipt, the signed-in owner as updatedBy, the revision moves', async t => {
const f = await devServer(t);
const r = await f.post();
assert.equal(r.status, 200, r.text);
assert.equal(r.json.route.model, 'sonnet');
assert.equal(r.json.route.harness, 'claude-code');
assert.equal(r.json.route.cap, 200000);
assert.equal(r.json.route.timeZone, 'Europe/Berlin');
assert.equal(r.json.route.reset, '00:00');
assert.equal(r.json.revision, 1);
assert.equal(r.json.note, NOT_STARTED_NOTE);
const doc = readRoute(f.root);
assert.equal(doc.updatedBy, 'local-dev-owner');
assert.equal(doc.cap.limit, 200000);
const lines = receipts(f.root);
assert.equal(lines.length, 1);
assert.equal(lines[0].by, 'local-dev-owner');
assert.equal(lines[0].revision, 1);
const again = await f.post({ ...BODY, harness: 'pi', revision: 1 });
assert.equal(again.status, 200);
assert.equal(receipts(f.root).length, 2);
for (const line of f.routeLogs) assert.match(line, /^\S+ (GET|POST) \/api\/triage\/route \d{3}$/);
});
test('a stale revision answers 409 with the stored revision and writes nothing', async t => {
const f = await devServer(t);
assert.equal((await f.post()).status, 200);
const stale = await f.post({ ...BODY, cap: 9 });
assert.equal(stale.status, 409);
assert.equal(stale.json.error, 'triage-route-stale');
assert.equal(stale.json.revision, 1);
assert.match(stale.json.message, /changed since this page read it/);
assert.equal(readRoute(f.root).cap.limit, 200000);
assert.equal(receipts(f.root).length, 1);
});
test('a model the catalogue does not offer, or one it marks unavailable, is refused and nothing is written', async t => {
const f = await devServer(t);
const unknown = await f.post({ ...BODY, model: 'opus' });
assert.equal(unknown.status, 400);
assert.equal(unknown.json.error, 'unknown-model');
const down = await f.post({ ...BODY, model: 'down' });
assert.equal(down.status, 503);
assert.equal(down.json.error, 'model-provider-unavailable');
for (const bad of [7, ['sonnet'], { key: 'sonnet' }, 'BAD KEY', '']) {
const r = await f.post({ ...BODY, model: bad });
assert.equal(r.status, 400, JSON.stringify(bad));
assert.equal(r.json.error, 'unknown-model', JSON.stringify(bad));
}
assert.equal(readRoute(f.root), null);
assert.deepEqual(receipts(f.root), []);
});
test('a harness the model is not compatible with, or one not installed, is refused', async t => {
const f = await devServer(t);
const wrong = await f.post({ ...BODY, model: 'glm-flash', harness: 'claude-code' });
assert.equal(wrong.status, 400);
assert.equal(wrong.json.error, 'model-harness-incompatible');
// Installed here is only pi: the catalogue's claude-code is not offered and not accepted.
const g = await devServer(t, { harnesses: () => ['pi'] });
assert.deepEqual((await g.get()).json.choices.models.find(m => m.key === 'sonnet').harnesses, ['pi']);
const off = await g.post({ ...BODY, harness: 'claude-code' });
assert.equal(off.status, 400);
assert.ok(['model-harness-incompatible', 'harness-off'].includes(off.json.error), off.text);
const bad = await f.post({ ...BODY, harness: 'Not A Name' });
assert.equal(bad.json.error, 'harness-off');
assert.equal(readRoute(f.root), null);
assert.equal(readRoute(g.root), null);
});
test('the installed harness check is the organization one: a compatible harness no module provides is harness-off', async t => {
const catalogue = READY(); catalogue.models[0].harnesses = ['claude-code', 'pi', 'ghost'];
// harnesses() lists ghost so the merged catalogue keeps it; the check gets the installed list without it.
let calls = 0;
const f = await devServer(t, { catalogues: async () => [catalogue], harnesses: () => (++calls % 2 === 1 ? ['claude-code', 'pi', 'ghost'] : ['claude-code', 'pi']) });
const r = await f.post({ ...BODY, harness: 'ghost' });
assert.equal(r.json.error, 'harness-off', r.text);
});
test('a catalogue that is not ready, or no organization bus answering, refuses the write and shows no choices', async t => {
for (const catalogues of [async () => [{ status: 'starting', models: [] }], async () => [{ status: 'bus-unavailable' }], async () => [], async () => { throw new Error(SECRET); }]) {
const f = await devServer(t, { catalogues });
const r = await f.get();
assert.equal(r.status, 200);
assert.deepEqual(r.json.choices.models, []);
assert.notEqual(r.json.choices.status, 'ready');
const w = await f.post();
assert.equal(w.status, 503);
assert.equal(w.json.error, 'model-provider-unavailable');
assert.equal(w.text.includes(SECRET) || r.text.includes(SECRET), false);
assert.equal(readRoute(f.root), null);
}
});
test('a missing or unknown time zone is refused; there is no default', async t => {
const f = await devServer(t);
const { timeZone, ...noZone } = BODY;
for (const body of [noZone, { ...BODY, timeZone: '' }, { ...BODY, timeZone: null }]) {
const r = await f.post(body);
assert.equal(r.status, 400, JSON.stringify(body));
assert.equal(r.json.error, 'triage-zone-required', JSON.stringify(body));
}
for (const zone of ['Mars/Olympus', 'not a zone', 42]) {
const r = await f.post({ ...BODY, timeZone: zone });
assert.equal(r.status, 400, String(zone));
assert.equal(r.json.error, 'triage-zone-unknown', String(zone));
}
assert.equal(readRoute(f.root), null);
const lower = await f.post({ ...BODY, timeZone: 'america/chicago' });
assert.equal(lower.status, 200, JSON.stringify(lower.json));
assert.equal(readRoute(f.root).cap.timeZone, 'America/Chicago', 'a zone in another case is stored in its canonical spelling');
assert.equal(lower.json.route.timeZone, 'America/Chicago');
});
test('a cap that is not a positive whole number is refused', async t => {
const f = await devServer(t);
for (const cap of [0, -1, 1.5, '1000', null, undefined, Number.MAX_SAFE_INTEGER + 1, true]) {
const body = { ...BODY, cap }; if (cap === undefined) delete body.cap;
const r = await f.post(body);
assert.equal(r.status, 400, String(cap));
assert.equal(r.json.error, 'triage-cap-invalid', String(cap));
}
assert.equal((await f.post({ ...BODY, cap: 1 })).status, 200, 'one token is a cap');
});
test('a body with another key, a bad revision, a wrong type or too many bytes is invalid-request', async t => {
const f = await devServer(t);
for (const body of [{ ...BODY, by: 'mallory' }, { ...BODY, revision: -1 }, { ...BODY, revision: '0' }, [BODY], 'null', '{', JSON.stringify({ ...BODY, model: 'a'.repeat(17000) })]) {
const r = await f.post(body);
assert.equal(r.status, 400, typeof body === 'string' ? body.slice(0, 20) : JSON.stringify(body).slice(0, 60));
assert.equal(r.json.error, 'invalid-request');
}
const { revision, ...noRevision } = BODY;
assert.equal((await f.post(noRevision)).json.error, 'invalid-request', 'the revision is required');
const text = await f.post(BODY, { ...f.write, 'content-type': 'text/plain' });
assert.equal(text.json.error, 'invalid-request');
assert.equal(readRoute(f.root), null);
});
test('both routes refuse a cross-origin request; a read may omit the metadata; a write needs both checks exactly once', async t => {
const f = await devServer(t);
// Read: a foreign Origin or cross-site Fetch Metadata is refused. Missing metadata is the repo GET convention.
const evil = await f.get({ ...f.read, origin: 'http://evil.example' });
assert.equal(evil.status, 403);
assert.match(evil.json.error, /^cross-origin/, 'the serve gate or the handler refuses it');
const bare = await f.get({ host: f.read.host });
assert.equal(bare.status, 200, 'a read with no Origin and no Fetch Metadata is allowed');
assert.equal((await f.get({ ...f.read, 'sec-fetch-site': 'none' })).status, 200, 'a typed address is allowed');
const cross = await f.get({ ...f.read, 'sec-fetch-site': 'cross-site' });
assert.equal(cross.status, 403);
assert.equal(cross.json.error, 'cross-site');
assert.equal((await f.get({ host: f.read.host, 'sec-fetch-site': 'same-site' })).status, 403);
assert.equal((await f.get({ ...f.read, origin: `http://127.0.0.1:${f.port}` })).status, 200, 'a matching Origin on a read is fine');
// Write: missing Origin, foreign Origin, missing or cross-site Fetch Metadata.
const { origin, ...noOrigin } = f.write;
const missing = await f.post(BODY, noOrigin);
assert.equal(missing.status, 403);
assert.match(missing.json.error, /^cross-origin/, 'the serve gate or the handler refuses it');
assert.equal((await f.post(BODY, { ...f.write, origin: 'http://evil.example' })).status, 403);
const { 'sec-fetch-site': _, ...noSite } = f.write;
assert.equal((await f.post(BODY, noSite)).status, 403);
assert.equal((await f.post(BODY, { ...f.write, 'sec-fetch-site': 'same-site' })).status, 403);
assert.equal(readRoute(f.root), null);
assert.deepEqual(receipts(f.root), []);
});
test('a LAN dev host reads the route with no Fetch Metadata and writes with its own Origin', async t => {
const f = await devServer(t, { devAllowHosts: ['lanbox'] });
const host = `lanbox:${f.port}`;
const r = await f.get({ host });
assert.equal(r.status, 200);
assert.equal(r.json.route, null);
const w = await f.post(BODY, { host, origin: `http://${host}`, 'content-type': 'application/json' });
assert.equal(w.status, 200, JSON.stringify(w.json));
assert.equal(readRoute(f.root).model, BODY.model);
const foreign = await f.post(BODY, { host, origin: 'http://evil.example', 'content-type': 'application/json' });
assert.equal(foreign.status, 403);
assert.equal(readRoute(f.root).revision, 1);
});
test('the handler holds the same-origin line itself: a doubled Origin or Sec-Fetch-Site is refused', async t => {
const root = mkdtempSync(join(tmpdir(), 'tr-unit-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
let handler;
triageRouteRoutes({ dataRoot: root, catalogues: async () => [READY()], harnesses: () => ['claude-code', 'pi'] })({ add: (method, path, fn) => { handler = fn; } });
const call = (method, headersDistinct, extra = {}) => new Promise(resolve => {
const res = { writeHead(status) { this.status = status; }, end(text) { resolve({ status: this.status, json: JSON.parse(text) }); } };
const req = { method, headersDistinct, headers: { 'content-type': 'application/json' }, async *[Symbol.asyncIterator]() { yield Buffer.from(JSON.stringify(BODY)); } };
handler(req, res, { owner: true, session: { provider: 'local-dev', sub: 'local-dev-owner' }, origin: 'https://console.example', send: true, ...extra });
});
const o = 'https://console.example';
for (const method of ['PUT', 'DELETE', 'PATCH']) assert.equal((await call(method, { origin: [o], 'sec-fetch-site': ['same-origin'] })).json.error, 'method-not-allowed', method);
assert.equal(readRoute(root), null);
assert.equal((await call('GET', { origin: [o, o], 'sec-fetch-site': ['same-origin'] })).json.error, 'cross-origin');
assert.equal((await call('GET', { 'sec-fetch-site': ['same-origin', 'same-origin'] })).json.error, 'cross-site');
assert.equal((await call('GET', { origin: ['https://evil.example'], 'sec-fetch-site': ['same-origin'] })).json.error, 'cross-origin');
assert.equal((await call('POST', { origin: [o, o], 'sec-fetch-site': ['same-origin'] })).json.error, 'cross-origin');
assert.equal((await call('POST', { origin: [o], 'sec-fetch-site': ['same-origin', 'same-origin'] })).json.error, 'cross-site');
assert.equal((await call('POST', { origin: [o], 'sec-fetch-site': ['same-origin'] }, { send: false })).json.error, 'read-only');
assert.equal((await call('GET', { 'sec-fetch-site': ['same-origin'] }, { owner: false })).json.error, 'human-required');
assert.equal((await call('POST', { origin: [o], 'sec-fetch-site': ['same-origin'] }, { owner: false })).json.error, 'human-required');
assert.equal((await call('POST', { origin: [o], 'sec-fetch-site': ['same-origin'] }, { session: { provider: 'oidc', sub: 'Owner Sub' } })).json.error, 'human-required', 'an owner with no recordable name is refused');
assert.equal(readRoute(root), null);
assert.equal((await call('POST', { origin: [o], 'sec-fetch-site': ['same-origin'] })).status, 200);
});
test('a read-only Console reads the route but refuses a write', async t => {
const f = await devServer(t, { send: null });
const r = await f.get();
assert.equal(r.status, 200);
assert.equal(r.json.send, false);
const w = await f.post();
assert.equal(w.status, 403);
assert.equal(w.json.error, 'read-only');
assert.equal(readRoute(f.root), null);
});
test('an SSO root owner reads and writes as their account; a granted user and a signed-out caller reach neither route', async t => {
const access = { accessVersion: 1, revision: 1,
users: [{ id: 'alice', home: 'local', subjects: [{ provider: 'writer', sub: 'alice-sub' }] }],
grants: [{ id: 'g-1', user: 'alice', business: 'demo', projects: ['*'], instances: ['*'], level: 'collaborator' }] };
const auth = await createSsoSession(t, { host: 'console.example', users: consoleAccess({ load: () => access, business: () => 'demo' }) });
const root = mkdtempSync(join(tmpdir(), 'tr-sso-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
const s = await startServer({ port: 0, sso: auth.sso, send: async () => ({}), log: () => {},
routes: [triageRouteRoutes({ dataRoot: root, catalogues: async () => [READY()], harnesses: () => ['claude-code', 'pi'] })] });
t.after(() => new Promise(r => s.close(r)));
const port = s.address().port;
const owner = auth.writeHeaders(await auth.signIn(port));
assert.equal((await send(port, TRIAGE_ROUTE_PATH, { headers: owner })).status, 200);
const saved = await send(port, TRIAGE_ROUTE_PATH, { method: 'POST', headers: owner, body: BODY });
assert.equal(saved.status, 200, saved.text);
assert.match(readRoute(root).updatedBy, /^u-[0-9a-f]{12}$/);
const alice = auth.writeHeaders(await auth.signIn(port, { sub: 'alice-sub' }));
const aliceRead = await send(port, TRIAGE_ROUTE_PATH, { headers: alice });
assert.equal(aliceRead.status, 403);
assert.ok(['owner-only', 'human-required'].includes(aliceRead.json.error), aliceRead.text);
assert.equal(aliceRead.text.includes('sonnet'), false, 'a granted user sees no route');
const aliceWrite = await send(port, TRIAGE_ROUTE_PATH, { method: 'POST', headers: alice, body: { ...BODY, revision: 1, cap: 7 } });
assert.equal(aliceWrite.status, 403);
const nobody = await send(port, TRIAGE_ROUTE_PATH, { headers: { host: 'console.example', 'sec-fetch-site': 'same-origin' } });
assert.equal(nobody.status, 401);
const nobodyWrite = await send(port, TRIAGE_ROUTE_PATH, { method: 'POST', headers: { host: 'console.example', origin: 'https://console.example', 'sec-fetch-site': 'same-origin', 'content-type': 'application/json' }, body: { ...BODY, revision: 1 } });
assert.equal(nobodyWrite.status, 401);
// Cross-origin as the owner, on the public name.
assert.equal((await send(port, TRIAGE_ROUTE_PATH, { method: 'POST', headers: { ...owner, origin: 'https://evil.example' }, body: { ...BODY, revision: 1 } })).status, 403);
assert.equal((await send(port, TRIAGE_ROUTE_PATH, { headers: { ...owner, origin: 'https://evil.example' } })).status, 403);
assert.equal(readRoute(root).revision, 1);
assert.equal(readRoute(root).cap.limit, 200000);
assert.equal(receipts(root).length, 1);
});
test('the tester listener reaches neither route', async t => {
const { startTesterSocket } = await import('../src/tester-socket.mjs');
const { testerRequest } = await import('../../cli/src/tester-http.mjs');
const root = mkdtempSync(join(tmpdir(), 'tr-tester-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
const s = await startTesterSocket({ dataRoot: root, log: () => {}, routes: [triageRouteRoutes({ dataRoot: root, catalogues: async () => [READY()], harnesses: () => ['claude-code', 'pi'] })] });
t.after(() => new Promise(r => s.close(r)));
for (const method of ['GET', 'POST']) {
const r = await testerRequest({ dataRoot: root, path: TRIAGE_ROUTE_PATH, method, body: method === 'POST' ? BODY : undefined });
assert.equal(r.status, 403, method);
}
assert.equal(readRoute(root), null);
});
test('a busy store and an unreadable store answer with fixed words and no path', async t => {
const f = await devServer(t);
assert.equal((await f.post()).status, 200);
const { lock, file } = routePaths(f.root);
writeFileSync(lock, JSON.stringify({ pid: process.pid, at: T }), { mode: 0o600 });
const busy = await f.post({ ...BODY, revision: 1 });
assert.equal(busy.status, 409);
assert.equal(busy.json.error, 'triage-route-busy');
assert.equal(busy.text.includes(f.root) || busy.text.includes('route.lock'), false);
rmSync(lock);
writeFileSync(file, '{not json', { mode: 0o600 });
const broken = await f.get();
assert.equal(broken.status, 503);
assert.equal(broken.json.error, 'triage-route-unreadable');
assert.equal(broken.text.includes(f.root) || broken.text.includes('route.json'), false);
const brokenWrite = await f.post({ ...BODY, revision: 1 });
assert.equal(brokenWrite.json.error, 'triage-route-unreadable');
assert.equal(brokenWrite.text.includes(f.root), false);
});
test('no response or log line carries the sentinel, the data root or a store path', async t => {
const f = await devServer(t);
const all = [await f.get(), await f.post(), await f.get(), await f.post({ ...BODY, cap: 0 }), await f.post({ ...BODY, model: 'opus' }), await f.post({ ...BODY, revision: 0 }), await f.post(BODY, { ...f.write, origin: 'http://evil.example' })];
const text = all.map(r => r.text).join('\n') + f.routeLogs.join('\n') + f.logs.join('\n');
for (const needle of [SECRET, f.root, 'route.json', 'route.lock', 'route-receipts', '/srv/', '/home/', 'cliproxy', 'anthropic']) assert.equal(text.includes(needle), false, needle);
});
test('mergeCatalogues: a key two catalogues disagree on is dropped; availability and harnesses must hold everywhere', () => {
const a = { status: 'ready', models: [model('sonnet'), model('glm', { harnesses: ['pi'] }), model('x', { available: false })] };
const b = { status: 'ready', models: [model('sonnet', { harnesses: ['pi'] }), model('glm', { upstream: 'other' }), model('x'), model('y', { available: false })] };
a.models.push(model('y'));
const merged = mergeCatalogues([a, b, { status: 'bus-unavailable' }], ['claude-code', 'pi']);
assert.equal(merged.status, 'ready');
assert.deepEqual(merged.models.map(m => [m.key, m.available, m.harnesses]), [['sonnet', true, ['pi']], ['x', false, ['claude-code', 'pi']], ['y', false, ['claude-code', 'pi']]]);
assert.deepEqual(mergeCatalogues([{ status: 'starting' }], ['pi']), { status: 'starting', models: [] });
assert.deepEqual(mergeCatalogues([], ['pi']), { status: 'unavailable', models: [] });
assert.deepEqual(mergeCatalogues([{ status: 'ready', models: [{ key: 'BAD KEY', harnesses: ['pi'] }, null] }], ['pi']).models, []);
});
test('installedHarnesses lists the harness modules on disk; ownerActor names the account or the dev owner', () => {
assert.deepEqual(installedHarnesses(), ['claude-code', 'pi']);
assert.equal(ownerActor({ account: 'u-0123456789ab', provider: 'writer', sub: 'x' }), 'u-0123456789ab');
assert.equal(ownerActor({ provider: 'local-dev', sub: 'local-dev-owner' }), 'local-dev-owner');
assert.equal(ownerActor({ provider: 'writer', sub: 'owner-sub' }), null);
assert.equal(ownerActor(null), null);
});
@@ -0,0 +1,262 @@
// #1873 triage-settings: Settings > Triage in a real browser. The owner sees "not set", picks a model,
// a harness from that model's list, a cap and a time zone, and saves in three clicks; a refusal is one
// line and keeps the fields; a stale revision offers a re-read; a second Save while one runs does
// nothing; anyone /api/config does not call the owner gets no Triage page and no route read.
import test from 'node:test';
import assert from 'node:assert/strict';
import { w15Fixture } from './w15-fixture.mjs';
const T = '2026-11-18T12:00:00.000Z';
const NOTE = 'Saving a route does not start Triage. The key and the wiring are separate.';
const choices = { status: 'ready', models: [
{ key: 'down', label: 'Down', available: false, harnesses: ['claude-code'] },
{ key: 'glm-flash', label: 'GLM Flash', available: true, harnesses: ['pi'] },
{ key: 'sonnet', label: 'Sonnet', available: true, harnesses: ['claude-code', 'pi'] },
] };
const unset = { route: null, routeMessage: 'No Triage route is set.', revision: 0, choices, send: true, note: NOTE };
const route = (over = {}) => ({ model: 'sonnet', harness: 'pi', cap: 200000, reset: '00:00', timeZone: 'Europe/Berlin', revision: 1, updatedAt: T, ...over });
const config = (over = {}) => ({ send: true, devOwner: true, sso: true, proxyAdmin: true, proxyAdminSend: true, ...over });
const scenario = (read = unset, extra = {}, cfg = {}) => ({ '/api/config': config(cfg), '/api/triage/route': read, ...extra });
const writes = b => b.evaluate('window.__writes.filter(w=>w.path==="/api/triage/route").map(w=>({method:w.method,body:JSON.parse(w.body)}))');
// The fixture answers in the page, so no resource timing exists: count the route reads in a wrapper.
const countReads = b => b.call('Page.addScriptToEvaluateOnNewDocument', { source: '(()=>{window.__routeReads=0;const prev=window.fetch;window.fetch=(i,init)=>{try{if(new URL(typeof i==="string"?i:i.url,location.href).pathname==="/api/triage/route"&&(init?.method??"GET")==="GET")window.__routeReads++;}catch{}return prev(i,init);};})()' });
const reads = b => b.evaluate('window.__routeReads');
const pick = (b, id, value) => b.evaluate(`(()=>{const el=document.getElementById(${JSON.stringify(id)});el.value=${JSON.stringify(value)};el.dispatchEvent(new Event('input',{bubbles:true}));el.dispatchEvent(new Event('change',{bubbles:true}));})()`);
const type = (b, id, value) => b.evaluate(`(()=>{const el=document.getElementById(${JSON.stringify(id)});el.value=${JSON.stringify(value)};el.dispatchEvent(new Event('input',{bubbles:true}));})()`);
const text = (b, id) => b.evaluate(`document.getElementById(${JSON.stringify(id)})?.textContent ?? null`);
const value = (b, id) => b.evaluate(`document.getElementById(${JSON.stringify(id)})?.value ?? null`);
const options = (b, id) => b.evaluate(`[...document.getElementById(${JSON.stringify(id)}).options].map(o=>[o.value,o.textContent,o.disabled])`);
async function fill(b, { model = 'sonnet', harness = 'claude-code', cap = '150000', zone = 'America/Chicago' } = {}) {
await pick(b, 'triage-route-model', model);
if (harness !== null) await pick(b, 'triage-route-harness', harness);
await type(b, 'triage-route-cap', cap);
await type(b, 'triage-route-zone', zone);
}
test('the owner sees Triage in Settings, "not set" before a first save, the catalogue choices and the note', { timeout: 90000 }, async t => {
const { f, b, wait } = await w15Fixture(t, { scenario: scenario() });
await countReads(b);
await b.navigate(f.base + '/#/settings');
await wait('document.querySelector(\'#settings-nav a[href="#/settings/triage"]\')');
await b.evaluate('document.querySelector(\'#settings-nav a[href="#/settings/triage"]\').click()');
await wait('document.getElementById("triage-route-stored")');
assert.match(await text(b, 'triage-route-stored'), /^Not set\. No Triage route is set\.$/);
assert.ok(await reads(b) >= 1, 'the owner page reads the route (the counter works)');
assert.equal(await text(b, 'triage-route-note'), NOTE);
assert.equal(await text(b, 'settings-page-title'), 'Triage');
assert.deepEqual(await options(b, 'triage-route-model'), [['', 'Choose a model', false], ['down', 'Down (down) (unavailable)', true], ['glm-flash', 'GLM Flash (glm-flash)', false], ['sonnet', 'Sonnet (sonnet)', false]]);
assert.deepEqual(await options(b, 'triage-route-harness'), [['', 'Choose a model first', false]]);
assert.equal(await value(b, 'triage-route-cap'), '', 'no default cap');
assert.equal(await value(b, 'triage-route-zone'), '', 'no default time zone');
assert.equal(await value(b, 'triage-route-reset'), '00:00');
assert.equal(await b.evaluate('document.getElementById("triage-route-reset").readOnly'), true);
assert.equal(await b.evaluate('document.querySelector(".settings-readonly")'), null);
assert.equal(await b.evaluate('document.getElementById("triage-route-catalogue")'), null);
// The harness list follows the model; a model with one harness picks it.
await pick(b, 'triage-route-model', 'glm-flash');
assert.deepEqual(await options(b, 'triage-route-harness'), [['', 'Choose a harness', false], ['pi', 'pi', false]]);
assert.equal(await value(b, 'triage-route-harness'), 'pi');
await pick(b, 'triage-route-model', 'sonnet');
assert.deepEqual((await options(b, 'triage-route-harness')).map(o => o[0]), ['', 'claude-code', 'pi']);
assert.equal(await value(b, 'triage-route-harness'), 'pi', 'a harness the new model takes is kept');
assert.deepEqual(await writes(b), []);
assert.deepEqual(await b.evaluate('window.__unexpected'), []);
assert.deepEqual(await b.evaluate('window.__errors'), []);
});
test('Save posts the fields and the read revision once, then shows the stored route; three clicks from Settings', { timeout: 90000 }, async t => {
const saved = route({ harness: 'claude-code', cap: 150000, timeZone: 'America/Chicago' });
const { f, b, wait } = await w15Fixture(t, { scenario: scenario(unset, { 'POST /api/triage/route': { body: { route: saved, revision: 1, note: NOTE } } }) });
await b.navigate(f.base + '/#/settings');
await wait('document.querySelector(\'#settings-nav a[href="#/settings/triage"]\')');
await b.evaluate('document.querySelector(\'#settings-nav a[href="#/settings/triage"]\').click()');
await wait('document.getElementById("triage-route-model")');
await fill(b, { zone: ' America/Chicago ' });
await b.evaluate('document.getElementById("triage-route-save").click()');
await wait('document.getElementById("triage-route-line")?.textContent==="Saved. Revision 1."');
assert.equal(await b.evaluate('document.activeElement?.id'), 'triage-route-line', 'focus lands on the saved line, not the page body');
assert.deepEqual(await writes(b), [{ method: 'POST', body: { model: 'sonnet', harness: 'claude-code', cap: 150000, timeZone: 'America/Chicago', revision: 0 } }]);
assert.match(await text(b, 'triage-route-stored'), /Stored route: sonnet on claude-code, cap 150,000 tokens a day, reset 00:00 America\/Chicago\. Revision 1/);
assert.equal(await value(b, 'triage-route-cap'), '150000');
// The next save carries the new revision.
await type(b, 'triage-route-cap', '9000');
await b.evaluate('document.getElementById("triage-route-save").click()');
await wait('window.__writes.filter(w=>w.path==="/api/triage/route").length===2');
assert.equal((await writes(b))[1].body.revision, 1);
assert.deepEqual(await b.evaluate('window.__unexpected'), []);
});
test('a stored route fills the fields; each refusal is one plain line and keeps the fields', { timeout: 90000 }, async t => {
const { f, b, wait } = await w15Fixture(t, { scenario: scenario({ ...unset, route: route(), routeMessage: undefined, revision: 1 }) });
await b.navigate(f.base + '/#/settings/triage');
await wait('document.getElementById("triage-route-model")?.value==="sonnet"');
assert.equal(await value(b, 'triage-route-harness'), 'pi');
assert.equal(await value(b, 'triage-route-cap'), '200000');
assert.equal(await value(b, 'triage-route-zone'), 'Europe/Berlin');
assert.match(await text(b, 'triage-route-stored'), /^Stored route: sonnet on pi, cap 200,000 tokens a day, reset 00:00 Europe\/Berlin\. Revision 1, saved 2026-11-18T12:00:00\.000Z\.$/);
for (const [error, message, status] of [['triage-zone-unknown', 'The time zone is not one this Console knows.', 400], ['model-harness-incompatible', 'That harness does not run this model.', 400], ['model-provider-unavailable', 'The model provider is not available now.', 503]]) {
await b.evaluate(`Object.assign(window.__scenario, {'POST /api/triage/route': {status: ${status}, body: {error: ${JSON.stringify(error)}, message: ${JSON.stringify(message)}}}})`);
await fill(b, { zone: 'Mars/Olympus', cap: '777' });
await b.evaluate('document.getElementById("triage-route-save").click()');
await wait(`document.getElementById("triage-route-line")?.textContent===${JSON.stringify(message)}`);
assert.equal(await value(b, 'triage-route-model'), 'sonnet');
assert.equal(await value(b, 'triage-route-harness'), 'claude-code');
assert.equal(await value(b, 'triage-route-cap'), '777');
assert.equal(await value(b, 'triage-route-zone'), 'Mars/Olympus');
assert.equal(await b.evaluate('document.getElementById("triage-route-reload")'), null, 'only a conflict offers a re-read');
assert.equal(await b.evaluate('document.getElementById("triage-route-save").disabled'), false);
}
assert.match(await text(b, 'triage-route-stored'), /Revision 1/, 'a refusal leaves the stored line alone');
// A refusal with no readable answer still says one line.
await b.evaluate(`Object.assign(window.__scenario, {'POST /api/triage/route': {status: 500, body: {}}})`);
await b.evaluate('document.getElementById("triage-route-save").click()');
await wait('document.getElementById("triage-route-line")?.textContent==="The route was refused. Nothing was saved."');
assert.equal(await value(b, 'triage-route-cap'), '777');
// Leaving the page and coming back reads the route again and keeps what was typed.
await b.evaluate('location.hash="#/settings/appearance"');
await wait('!document.getElementById("triage-route")');
await b.evaluate('location.hash="#/settings/triage"');
await wait('document.getElementById("triage-route-cap")');
assert.equal(await value(b, 'triage-route-cap'), '777');
assert.equal(await value(b, 'triage-route-zone'), 'Mars/Olympus');
assert.match(await text(b, 'triage-route-stored'), /Revision 1/);
});
test('a stale revision says so, keeps the fields, and Reload route reads the stored route again', { timeout: 90000 }, async t => {
const message = 'The Triage route changed since this page read it. Reload the route, then save again.';
const { f, b, wait } = await w15Fixture(t, { scenario: scenario(unset, { 'POST /api/triage/route': { status: 409, body: { error: 'triage-route-stale', message, revision: 2 } } }) });
await b.navigate(f.base + '/#/settings/triage');
await wait('document.getElementById("triage-route-model")');
await fill(b, { cap: '5000' });
await b.evaluate('document.getElementById("triage-route-save").click()');
await wait(`document.getElementById("triage-route-line")?.textContent===${JSON.stringify(message)}`);
assert.equal(await value(b, 'triage-route-cap'), '5000');
assert.equal(await b.evaluate('document.activeElement?.id'), 'triage-route-reload');
await b.evaluate(`window.__scenario['/api/triage/route'] = ${JSON.stringify({ ...unset, route: route({ revision: 2, model: 'glm-flash', cap: 42 }), routeMessage: undefined, revision: 2 })}`);
await b.evaluate('document.getElementById("triage-route-reload").click()');
await wait('document.getElementById("triage-route-model")?.value==="glm-flash"');
assert.equal(await value(b, 'triage-route-cap'), '42');
assert.match(await text(b, 'triage-route-stored'), /Revision 2/);
assert.equal(await b.evaluate('document.getElementById("triage-route-reload")'), null);
await b.evaluate(`window.__scenario['POST /api/triage/route'] = {body: {route: ${JSON.stringify(route({ revision: 3, model: 'glm-flash', cap: 42 }))}, revision: 3, note: ${JSON.stringify(NOTE)}}}`);
await b.evaluate('document.getElementById("triage-route-save").click()');
await wait('document.getElementById("triage-route-line")?.textContent==="Saved. Revision 3."');
assert.equal((await writes(b)).at(-1).body.revision, 2, 'the save after a re-read carries the stored revision');
});
test('an untouched page follows the store; typed fields keep the revision they were read at, so a later save is stale', { timeout: 90000 }, async t => {
const stored = (over, revision) => ({ ...unset, route: route({ revision, ...over }), routeMessage: undefined, revision });
const message = 'The Triage route changed since this page read it. Reload the route, then save again.';
const { f, b, wait } = await w15Fixture(t, { scenario: scenario(stored({}, 1), { 'POST /api/triage/route': { status: 409, body: { error: 'triage-route-stale', message, revision: 3 } } }) });
const away = async () => {
await b.evaluate('location.hash="#/settings/appearance"');
await wait('!document.getElementById("triage-route")');
await b.evaluate('location.hash="#/settings/triage"');
await wait('document.getElementById("triage-route-cap")');
};
const dot = 'document.querySelector(\'#settings-nav a[href="#/settings/triage"] .settings-dot\') !== null';
await b.navigate(f.base + '/#/settings/triage');
await wait('document.getElementById("triage-route-model")?.value==="sonnet"');
// Saved elsewhere while this page sat untouched: the page shows the new route, with no unsaved mark.
await b.evaluate(`window.__scenario['/api/triage/route'] = ${JSON.stringify(stored({ model: 'glm-flash', cap: 42 }, 2))}`);
await away();
assert.equal(await value(b, 'triage-route-model'), 'glm-flash');
assert.equal(await value(b, 'triage-route-cap'), '42');
assert.match(await text(b, 'triage-route-stored'), /Revision 2/);
assert.equal(await b.evaluate(dot), false);
// Typed at revision 2, then saved elsewhere again: the typed fields stay and the save sends revision 2.
await type(b, 'triage-route-cap', '9000');
await b.evaluate(`window.__scenario['/api/triage/route'] = ${JSON.stringify(stored({ model: 'sonnet', cap: 1 }, 3))}`);
await away();
assert.equal(await value(b, 'triage-route-model'), 'glm-flash');
assert.equal(await value(b, 'triage-route-cap'), '9000');
assert.match(await text(b, 'triage-route-stored'), /Revision 3/);
assert.equal(await b.evaluate(dot), true, 'the typed fields are marked unsaved');
await b.evaluate('document.getElementById("triage-route-save").click()');
await wait(`document.getElementById("triage-route-line")?.textContent===${JSON.stringify(message)}`);
const last = (await writes(b)).at(-1).body;
assert.equal(last.revision, 2, 'the save carries the revision the typed fields were read at, not the newest read');
assert.equal(last.model, 'glm-flash');
assert.equal(last.cap, 9000);
});
test('a second Save while one runs does nothing', { timeout: 90000 }, async t => {
const { f, b, wait } = await w15Fixture(t, { scenario: scenario(unset, { 'POST /api/triage/route': { body: { route: route(), revision: 1, note: NOTE } } }) });
await b.navigate(f.base + '/#/settings/triage');
await wait('document.getElementById("triage-route-model")');
// Hold the POST until the test lets it go.
await b.evaluate('(()=>{const prev=window.fetch;window.__posts=0;window.fetch=(input,init)=>init?.method==="POST"?(window.__posts++,new Promise(r=>{window.__release=()=>r(prev(input,init));})):prev(input,init);})()');
await fill(b);
await b.evaluate('document.getElementById("triage-route-save").click()');
await wait('typeof window.__release==="function"');
assert.equal(await text(b, 'triage-route-save'), 'Saving…');
assert.equal(await b.evaluate('document.getElementById("triage-route-save").disabled'), true);
// A click on the disabled button, a form submit (Enter) and a direct requestSubmit all do nothing.
await b.evaluate('document.getElementById("triage-route-save").click()');
await b.evaluate('document.getElementById("triage-route-form").requestSubmit()');
await b.evaluate('document.getElementById("triage-route-form").dispatchEvent(new Event("submit",{bubbles:true,cancelable:true}))');
assert.equal(await b.evaluate('window.__writes.filter(w=>w.path==="/api/triage/route").length'), 0, 'the held POST is not recorded yet');
assert.equal(await b.evaluate('window.__posts'), 1, 'no second POST left the page');
await b.evaluate('window.__release()');
await wait('document.getElementById("triage-route-line")?.textContent==="Saved. Revision 1."');
assert.equal((await writes(b)).length, 1);
});
test('a read-only Console shows the route with Save off and posts nothing', { timeout: 90000 }, async t => {
const { f, b, wait } = await w15Fixture(t, { scenario: scenario({ ...unset, send: false }, {}, { send: false, proxyAdminSend: false }) });
await b.navigate(f.base + '/#/settings/triage');
await wait('document.getElementById("triage-route-save")');
assert.equal(await b.evaluate('document.getElementById("triage-route-save").disabled'), true);
assert.match(await text(b, 'triage-route-readonly'), /read-only/);
assert.equal(await b.evaluate('document.querySelector(".settings-readonly")'), null, 'the panel says read-only once, from the route\'s own answer');
await b.evaluate('document.getElementById("triage-route-form").dispatchEvent(new Event("submit",{bubbles:true,cancelable:true}))');
assert.deepEqual(await writes(b), []);
});
test('a catalogue that is not ready says so and offers no model', { timeout: 90000 }, async t => {
const { f, b, wait } = await w15Fixture(t, { scenario: scenario({ ...unset, choices: { status: 'bus-unavailable', models: [] } }) });
await b.navigate(f.base + '/#/settings/triage');
await wait('document.getElementById("triage-route-catalogue")');
assert.match(await text(b, 'triage-route-catalogue'), /No organization bus offered a model catalogue \(bus-unavailable\)/);
assert.deepEqual(await options(b, 'triage-route-model'), [['', 'Choose a model', false]]);
});
test('a non-owner sees no Triage section and the page reads no route', { timeout: 90000 }, async t => {
const { f, b, wait } = await w15Fixture(t, { scenario: scenario(unset, {}, { proxyAdmin: undefined, proxyAdminSend: undefined, user: true }) });
await countReads(b);
await b.navigate(f.base + '/#/settings');
await wait('document.querySelector(\'#settings-nav a[href="#/settings/appearance"]\')');
assert.equal(await b.evaluate('document.querySelector(\'#settings-nav a[href="#/settings/triage"]\')'), null);
await b.navigate(f.base + '/#/settings/triage');
await wait('document.querySelector(".nf")');
assert.equal(await b.evaluate('document.getElementById("triage-route")'), null);
assert.equal(await reads(b), 0, 'no route read for a non-owner');
assert.deepEqual(await writes(b), []);
});
// Shots for the build report: run with MOSAIC_UX_SHOTS=<dir>. Not set, filled and one refusal, at 390
// and 1440, light and dark.
test('shots', { timeout: 300000, skip: !process.env.MOSAIC_UX_SHOTS }, async t => {
const { f, b, wait, capture } = await w15Fixture(t, { slice: 'triage', scenario: scenario(unset, { 'POST /api/triage/route': { status: 400, body: { error: 'triage-zone-unknown', message: 'The time zone is not one this Console knows.' } } }) });
const results = [];
for (const scheme of ['light', 'dark']) {
await b.call('Emulation.setEmulatedMedia', { features: [{ name: 'prefers-color-scheme', value: scheme }] });
for (const width of [390, 1440]) {
await b.viewport(width, width === 390 ? 844 : 900);
// A fresh document each round, so the kept fields of the last round do not carry over.
await b.navigate(`${f.base}/?shot=${scheme}-${width}#/settings/triage`);
await wait('document.getElementById("triage-route-model") && !!window.MosaicAppearance');
await b.evaluate(`window.MosaicAppearance.ready.then(() => window.MosaicAppearance.set({ mode: ${JSON.stringify(scheme)} }))`);
// No sideways scroll at either width.
assert.equal(await b.evaluate('document.documentElement.scrollWidth <= document.documentElement.clientWidth'), true, `${scheme} ${width} scrolls sideways`);
results.push(await capture(`notset-${scheme}`, width, '#/settings/triage'));
await fill(b);
results.push(await capture(`filled-${scheme}`, width, '#/settings/triage'));
await type(b, 'triage-route-zone', 'Mars/Olympus');
await b.evaluate('document.getElementById("triage-route-save").click()');
await wait('document.getElementById("triage-route-line")?.textContent.includes("time zone")');
results.push(await capture(`refusal-${scheme}`, width, '#/settings/triage'));
}
}
for (const r of results) { assert.deepEqual(r.errors, [], r.full); assert.deepEqual(r.unexpected, [], r.full); }
});
+205
View File
@@ -0,0 +1,205 @@
// #1873 triage-cap: Triage's usage file and its daily cap. One line per turn in
// <dataRoot>/triage/usage.jsonl, held to the same file checks as the route store; the cap is the
// daily-quota plugin's own rule over those lines, with the stored cap as its one leaf.
import { test } from "node:test";
import assert from "node:assert/strict";
import { chmodSync, linkSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, symlinkSync, writeFileSync } from "node:fs";
import { execFileSync } from "node:child_process";
import { join } from "node:path";
import { mayLaunch, report } from "../../plugins/limit-check/daily-quota/index.mjs";
import {
USAGE_ACCOUNT, USAGE_FIELDS, appendUsage, capState, delta, limitsFor, nextReset, readUsage, snapshot, usagePath,
} from "../src/triage-usage.mjs";
const scratch = () => mkdtempSync(join(process.env.TMPDIR, "tu-"));
const fileIn = () => {
const dir = join(scratch(), "triage");
mkdirSync(dir, { mode: 0o700 });
return join(dir, "usage.jsonl");
};
const line = (over = {}) => ({ at: "2026-11-18T12:00:00.000Z", run: "triage-20261118120000", turn: "owner-0123456789ab", status: "known", input: 10, output: 5, cacheRead: 2, cacheWrite: 1, ...over });
const unknownLine = (over = {}) => line({ status: "unknown", input: null, output: null, cacheRead: null, cacheWrite: null, ...over });
const cap = (over = {}) => ({ limit: 100, since: "2026-10-01T00:00:00.000Z", reset: "00:00", timeZone: "UTC", ...over });
const ms = (s) => Date.parse(s);
test("the usage file is <dataRoot>/triage/usage.jsonl and a line holds exactly its eight fields", () => {
assert.equal(usagePath("/d"), "/d/triage/usage.jsonl");
assert.deepEqual(USAGE_FIELDS, ["at", "run", "turn", "status", "input", "output", "cacheRead", "cacheWrite"]);
assert.equal(USAGE_ACCOUNT, "triage");
});
test("each append writes one line, 0600, in field order; a torn last line is closed off before the next line", () => {
const file = fileIn();
appendUsage(file, line());
appendUsage(file, unknownLine({ turn: "owner-0123456789ac" }));
assert.equal(lstatSync(file).mode & 0o777, 0o600);
const text = readFileSync(file, "utf8");
assert.equal(text.split("\n").length, 3);
assert.deepEqual(Object.keys(JSON.parse(text.split("\n")[0])), USAGE_FIELDS);
// A crash in a write leaves a torn line. The next append starts on a line of its own.
writeFileSync(file, '{"at":"2026-11-18T12:0', { flag: "a" });
appendUsage(file, line({ turn: "owner-0123456789ad" }));
const lines = readFileSync(file, "utf8").split("\n");
assert.equal(lines.length, 5);
assert.equal(JSON.parse(lines[3]).turn, "owner-0123456789ad");
// A line that does not match its shape is never written.
for (const bad of [{ ...line(), extra: 1 }, line({ at: "yesterday" }), line({ input: -1 }), line({ status: "unknown" }), unknownLine({ input: 3 }), line({ turn: "a b" }), line({ run: "" })]) {
assert.throws(() => appendUsage(file, bad), (e) => e.code === "triage-usage-unreadable", JSON.stringify(bad));
}
assert.equal(readFileSync(file, "utf8").split("\n").length, 5);
});
test("a narrow umask still leaves a new usage file at 0600", () => {
const file = fileIn();
const old = process.umask(0o277);
try { appendUsage(file, line()); } finally { process.umask(old); }
assert.equal(lstatSync(file).mode & 0o777, 0o600);
});
// Each unsafe state refuses both the read and the append, and the append leaves the target alone.
function unsafe(make) {
const file = fileIn();
const opts = make(file) ?? {};
return { file, opts };
}
test("a wrong mode, a wrong owner, a symlink, a hard link or a non-file refuses the read and the append", () => {
const cases = {
"mode 0644": (f) => { appendUsage(f, line()); chmodSync(f, 0o644); },
"mode 0400": (f) => { appendUsage(f, line()); chmodSync(f, 0o400); },
"owner": (f) => { appendUsage(f, line()); return { uid: process.getuid() + 1 }; },
"symlink": (f) => { const real = join(f, "..", "real.jsonl"); writeFileSync(real, "", { mode: 0o600 }); symlinkSync(real, f); },
"dangling symlink": (f) => { symlinkSync(join(f, "..", "absent.jsonl"), f); },
"hard link": (f) => { appendUsage(f, line()); linkSync(f, join(f, "..", "other.jsonl")); },
"directory": (f) => { mkdirSync(f, { mode: 0o700 }); },
"fifo": (f) => { execFileSync("mkfifo", ["-m", "600", f]); },
};
for (const [label, make] of Object.entries(cases)) {
const { file, opts } = unsafe(make);
const before = lstatSync(file);
assert.throws(() => readUsage(file, opts), (e) => e.code === "triage-usage-unreadable", `read: ${label}`);
assert.throws(() => appendUsage(file, line(), opts), (e) => e.code === "triage-usage-unreadable", `append: ${label}`);
const after = lstatSync(file);
assert.deepEqual([after.size, after.mode, after.ino], [before.size, before.mode, before.ino], `untouched: ${label}`);
}
// The dangling link's target is not made.
const { file } = unsafe((f) => { symlinkSync(join(f, "..", "absent.jsonl"), f); });
try { appendUsage(file, line()); } catch {}
assert.throws(() => lstatSync(join(file, "..", "absent.jsonl")), (e) => e.code === "ENOENT");
});
test("a usage file over the size bound refuses before it is parsed", () => {
const file = fileIn();
appendUsage(file, line());
appendUsage(file, line({ turn: "owner-0123456789ac" }));
const size = lstatSync(file).size;
assert.equal(readUsage(file, { maxBytes: size }).records.length, 2);
assert.throws(() => readUsage(file, { maxBytes: size - 1 }), (e) => e.code === "triage-usage-unreadable");
});
test("no file reads as no turns; good lines read back as records for the triage account", () => {
const file = fileIn();
assert.deepEqual(readUsage(file), { records: [], bad: [] });
appendUsage(file, line());
appendUsage(file, unknownLine({ turn: "owner-0123456789ac" }));
const { records, bad } = readUsage(file);
assert.deepEqual(bad, []);
assert.deepEqual(records, [{ account: "triage", ...line() }, { account: "triage", ...unknownLine({ turn: "owner-0123456789ac" }) }]);
});
test("a torn last line is refused and counts as one unknown turn; a wrong-shaped line the same", () => {
const file = fileIn();
appendUsage(file, line());
writeFileSync(file, '{"at":"2026-11-18T13:00:00.000Z","run":"triage-2026', { flag: "a" });
const torn = readUsage(file);
assert.equal(torn.records.length, 2);
assert.deepEqual(torn.bad.map((b) => b.line), [2]);
const [, b] = torn.records;
assert.deepEqual([b.status, b.run, b.turn, b.input, b.output, b.cacheRead, b.cacheWrite], ["unknown", null, "line-2", null, null, null, null]);
assert.ok(!Number.isNaN(Date.parse(b.at)), "a torn last line takes the file's change time");
const [r] = report({ limits: limitsFor(cap()), records: torn.records, now: "2026-11-18T23:00:00.000Z" });
assert.deepEqual([r.count, r.unknownTurns], [18, 1]);
// Closed off by the next append, it stays one unknown turn, dated by the line after it.
appendUsage(file, line({ at: "2026-11-18T14:00:00.000Z", turn: "owner-0123456789ac", input: 1, output: 0, cacheRead: 0, cacheWrite: 0 }));
writeFileSync(file, JSON.stringify({ ...line(), input: "9" }) + "\n", { flag: "a" });
const later = readUsage(file);
assert.deepEqual(later.bad.map((x) => x.line), [2, 4]);
assert.equal(later.records[1].at, "2026-11-18T14:00:00.000Z");
const [again] = report({ limits: limitsFor(cap()), records: later.records, now: "2026-11-18T23:00:00.000Z" });
assert.deepEqual([again.count, again.unknownTurns], [19, 2]);
});
test("the limits copy the organization shape: one daily leaf for the triage account", () => {
assert.deepEqual(limitsFor(cap()), { tokens: { daily: { triage: cap() } } });
const records = [{ account: "triage", ...line({ input: 100, output: 0, cacheRead: 0, cacheWrite: 0 }) }];
assert.deepEqual(mayLaunch({ limits: limitsFor(cap()), subject: { account: "triage" }, records, now: "2026-11-18T13:00:00.000Z" }).allow, false);
});
test("the cap state: under, at and over the cap, unknown turns adding nothing, and the cap's since", () => {
const now = ms("2026-11-18T18:00:00.000Z");
const recs = (...counts) => counts.map((n, i) => ({ account: "triage", ...line({ turn: `owner-00000000000${i}`, input: n, output: 0, cacheRead: 0, cacheWrite: 0 }) }));
const under = capState({ cap: cap(), records: recs(40, 59), now });
assert.equal(under.allow, true);
assert.deepEqual(under.usageToday, { count: 99, input: 99, output: 0, cacheRead: 0, cacheWrite: 0, unknownTurns: 0 });
assert.equal(under.resetsAt, "2026-11-19T00:00:00.000Z");
const at = capState({ cap: cap(), records: recs(40, 60), now });
assert.equal(at.allow, false);
assert.equal(at.usageToday.count, 100);
const withUnknown = capState({ cap: cap(), records: [...recs(40), { account: "triage", ...unknownLine({ turn: "owner-00000000000f" }) }], now });
assert.deepEqual([withUnknown.allow, withUnknown.usageToday.count, withUnknown.usageToday.unknownTurns], [true, 40, 1]);
// Lines before the cap's since or before today's reset do not count.
const old = [{ account: "triage", ...line({ at: "2026-11-17T23:59:59.999Z", input: 500, output: 0, cacheRead: 0, cacheWrite: 0 }) }];
assert.equal(capState({ cap: cap(), records: old, now }).usageToday.count, 0);
const since = [{ account: "triage", ...line({ at: "2026-11-18T10:00:00.000Z", input: 500, output: 0, cacheRead: 0, cacheWrite: 0 }) }];
assert.equal(capState({ cap: cap({ since: "2026-11-18T11:00:00.000Z" }), records: since, now }).allow, true);
// A cap of 0 takes no turn, as the plugin's own rule says.
assert.equal(capState({ cap: cap({ limit: 0 }), records: [], now }).allow, false);
});
test("the reset is 00:00 in the stored zone: the next one is the time shown and the day turns over there", () => {
const z = (timeZone) => cap({ timeZone });
assert.equal(new Date(nextReset(z("UTC"), ms("2026-11-18T18:00:00.000Z"))).toISOString(), "2026-11-19T00:00:00.000Z");
assert.equal(new Date(nextReset(z("UTC"), ms("2026-11-18T00:00:00.000Z"))).toISOString(), "2026-11-19T00:00:00.000Z", "at the reset, the next is a day on");
assert.equal(new Date(nextReset(z("UTC"), ms("2026-11-18T23:59:59.999Z"))).toISOString(), "2026-11-19T00:00:00.000Z");
assert.equal(new Date(nextReset(z("Asia/Kolkata"), ms("2026-11-18T18:00:00.000Z"))).toISOString(), "2026-11-18T18:30:00.000Z");
// A since after now: the reset after it.
assert.equal(new Date(nextReset(cap({ since: "2026-11-19T06:00:00.000Z" }), ms("2026-11-18T18:00:00.000Z"))).toISOString(), "2026-11-20T00:00:00.000Z");
});
test("a DST day: America/Chicago falls back on 2026-11-01 (25 hours) and springs forward on 2026-03-08 (23 hours)", () => {
const chicago = cap({ timeZone: "America/Chicago", since: "2026-01-01T00:00:00.000Z", limit: 50 });
// Fall back: the day runs from 05:00Z (00:00 CDT) to 06:00Z next day (00:00 CST).
assert.equal(new Date(nextReset(chicago, ms("2026-11-01T12:00:00.000Z"))).toISOString(), "2026-11-02T06:00:00.000Z");
assert.equal(new Date(nextReset(chicago, ms("2026-11-01T05:30:00.000Z"))).toISOString(), "2026-11-02T06:00:00.000Z", "inside the repeated hour's day");
const fall = [
{ account: "triage", ...line({ at: "2026-11-01T04:59:59.999Z", turn: "owner-000000000001", input: 30, output: 0, cacheRead: 0, cacheWrite: 0 }) },
{ account: "triage", ...line({ at: "2026-11-01T05:00:00.000Z", turn: "owner-000000000002", input: 30, output: 0, cacheRead: 0, cacheWrite: 0 }) },
{ account: "triage", ...line({ at: "2026-11-02T05:30:00.000Z", turn: "owner-000000000003", input: 30, output: 0, cacheRead: 0, cacheWrite: 0 }) },
];
const lateFall = capState({ cap: chicago, records: fall, now: ms("2026-11-02T05:59:59.999Z") });
assert.deepEqual([lateFall.allow, lateFall.usageToday.count, lateFall.resetsAt], [false, 60, "2026-11-02T06:00:00.000Z"], "the 25th hour still counts the day");
const nextDay = capState({ cap: chicago, records: fall, now: ms("2026-11-02T06:00:00.000Z") });
assert.deepEqual([nextDay.allow, nextDay.usageToday.count, nextDay.resetsAt], [true, 0, "2026-11-03T06:00:00.000Z"]);
// Spring forward: the day runs from 06:00Z (00:00 CST) to 05:00Z next day (00:00 CDT).
assert.equal(new Date(nextReset(chicago, ms("2026-03-08T12:00:00.000Z"))).toISOString(), "2026-03-09T05:00:00.000Z");
const spring = [{ account: "triage", ...line({ at: "2026-03-08T06:00:00.000Z", input: 60, output: 0, cacheRead: 0, cacheWrite: 0 }) }];
assert.equal(capState({ cap: chicago, records: spring, now: ms("2026-03-09T04:59:59.999Z") }).allow, false);
assert.equal(capState({ cap: chicago, records: spring, now: ms("2026-03-09T05:00:00.000Z") }).allow, true);
});
test("the turn delta keeps the bus's rules: new entries summed, a rewritten or missing baseline unknown, no new usage unknown", () => {
const e = (input) => ({ input, output: 1, cacheWrite: 0, cacheRead: 0 });
const begin = snapshot({ status: "known", entries: { a: e(5) } });
assert.deepEqual(delta(begin, snapshot({ status: "known", entries: { a: e(5), b: e(7), c: e(1) } })), { status: "known", input: 8, output: 2, cacheWrite: 0, cacheRead: 0 });
assert.deepEqual(delta(begin, snapshot({ status: "known", entries: { a: e(5) } })), { status: "unknown", code: "no-new-usage" });
assert.deepEqual(delta(begin, snapshot({ status: "known", entries: { a: e(6), b: e(1) } })), { status: "unknown", code: "transcript-rewritten" });
assert.deepEqual(delta(begin, snapshot({ status: "known", entries: { b: e(1) } })), { status: "unknown", code: "transcript-rewritten" });
assert.deepEqual(delta(null, snapshot({ status: "known", entries: { b: e(1) } })), { status: "unknown", code: "baseline-unknown" });
assert.deepEqual(delta(snapshot({ status: "unknown", code: "transcript-size" }), snapshot({ status: "known", entries: {} })), { status: "unknown", code: "baseline-unknown" });
assert.deepEqual(delta(begin, snapshot({ status: "unknown", code: "transcript-parse" })), { status: "unknown", code: "transcript-parse" });
assert.deepEqual(snapshot({ status: "unknown", code: "anything" }), { status: "unknown", code: "source-unknown" });
assert.deepEqual(snapshot({ status: "known", entries: { a: { input: -1, output: 0, cacheWrite: 0, cacheRead: 0 } } }), { status: "unknown", code: "source-invalid" });
assert.deepEqual(snapshot({ status: "known", entries: { a: { ...e(1), extra: 0 } } }), { status: "unknown", code: "source-invalid" });
assert.deepEqual(snapshot(null), { status: "unknown", code: "source-invalid" });
const big = { status: "known", entries: { a: e(Number.MAX_SAFE_INTEGER), b: e(1) } };
assert.deepEqual(delta(snapshot({ status: "known", entries: {} }), snapshot(big)), { status: "unknown", code: "usage-overflow" });
});