Compare commits

..
Author SHA1 Message Date
jason.woltje 66ef78d69e Give the stack-release rehearsal copies the model-proxy files config.mjs now loads
config.mjs imports the ride refusal lines from packages/model-proxy/src/settings.mjs, and its
common.mjs reads packages/model-proxy/PIN at load. The real stack copy already carries both
(COPY_PATHS). The test's hand-built rehearsal copies did not, so 22 cases failed scratch-config.
2026-11-21 22:45:57 -06:00
jason.woltje a0ec8e028c Fix the proxy-ride review findings
Review findings 1-4 (all P3). A ride file removed while validate runs now
refuses proxy-ride-invalid, not a TypeError stack. The config.mjs comment and
docs/TOOLS.md now say an owner problem refuses every verb on the rider while
proxy-ride.json exists, that the owner should run with its own config path
because the first side to lock fixes the lock identity, and that run-task.sh
still shows a ride refusal as a parse error. A new test fails if container-cli
stops passing ride: true: the stand-in launch step answers with the proxy it
was given.
2026-11-21 22:45:57 -06:00
jason.woltje 8c5764fd03 Let one instance's worker runs ride another instance's model proxy
Jason approved five items on 2026-11-21 (option C, Q4 first slice, worker
runs only). <configDir>/proxy-ride.json {"version":1,"instance":"NAME"}
names an instance of the same OS user. mosaic proxy ride --instance NAME
writes it and mosaic proxy ride --off removes it; both are human only.

The config loader resolves the owner's proxy.json through the owner's own
config and data root claim and adds proxy.owner. A worker run reads the
owner's management key and model on/off list, and writes its worker key
under the owner's lock, fence and state dir, with the rider's own key tag.
loadSystem drops a ridden proxy unless a worker run asks, so buses, the
Console and the proxy unit see no proxy for a rider.

New refusals: proxy-ride-invalid (bad file, a path, its own instance, a
shared data root), proxy-ride-owner-missing (no valid owner proxy, or the
owner rides too), proxy-ride-conflict (proxy.json and a ride file).
2026-11-21 22:45:57 -06:00
jason.woltje 6f0427024a Say every cause of an unknown worker model standing in the view's comment 2026-11-21 21:42:35 -06:00
jason.woltje 538fe12829 Read the config fresh for the Worker model card, and run its view alongside the status read
Review round for worker-model-setup (S6):
- The card reads execution from mosaic-config.mjs validate on each status read, so a config
  edit shows without a Console restart.
- status() starts the worker view first and awaits it last; status({ worker: false }) skips it,
  and the Triage snapshot passes that.
- The card says why the model list is not known: no organization, the model service off
  everywhere, or not read.
- The "config always wins" line no longer speaks of a choice made here.
- The standing 'claude' is now 'claude-model', since display.js may not key by a provider id.
2026-11-21 21:35:35 -06:00
jason.woltje 5f4ccd0326 Show the worker model on Model setup, and let the launch read a saved choice when the config names none
A Worker model card on Settings > Model setup (owner only, read only) says which model worker
runs use, that the config names it and wins, and lists only switched-on, worker-harness, non-Claude
models from the organizations' catalogues. The view rides the existing owner-gated status read.

A small store holds a worker model choice, guarded like the Triage route store. The launch reads
it only when the config names no model, holds it to the same Claude, switched-off and served checks,
and passes the checked model to compose. The config schema still requires a model, and no save
route is built; both are asks in the build report.
2026-11-21 20:02:13 -06:00
jason.woltje 1d39235a8b Keep an empty first field when the gate test reads its compose record 2026-11-21 19:02:01 -06:00
jason.woltje 13763db199 Test that a container-cli probe writes worker output with the key taken out
Review finding 5 (P3): no test failed if container-cli dropped its redaction. The test hook now
matches the resolved URL, since container-cli imports './worker-proxy.mjs'.
2026-11-21 18:17:27 -06:00
jason.woltje 717a8b9624 Run workers through the local model proxy with a Mosaic-issued key
A host worker launch (run-task.sh, mosaic-task.mjs, verify.sh, the release
health check) now checks the model, sweeps stale launches, issues or re-adds
the worker key, starts a relay to the proxy socket and mounts it at
/run/mosaic-proxy. The key enters the container only as ANTHROPIC_API_KEY and
is redacted from stderr.txt, the response and the run cause. The run record
keeps the key id only (modelKeyId).

The pi auth mount, the ZAI_API_KEY pass-through, the caller's
ANTHROPIC_API_KEY, the worker credential check, the auth.sh worker line and
the agent.sh account choice are gone. The harness name comes from
execution.adapter, and a Claude model never runs on pi. RELEASE is 0.0.14.
2026-11-21 17:54:33 -06:00
jason.woltje e5d1a37c9a Test that a task record with a container name keeps its worker key 2026-11-21 16:34:00 -06:00
jason.woltje 95a6be3b63 Hold a worker key for a container whose launcher died, and re-add a refused fast-path key under the lock 2026-11-21 16:34:00 -06:00
jason.woltje 882c12a136 Give the Triage key module a worker purpose, a lock-free launch path and a retire guard 2026-11-21 16:34:00 -06:00
jason.woltje 0aac96dd15 Stop naming a lost proxy for an idle close, and test that an error ends both sides
Re-check of bab5f47d9 found two minor things; this fixes them and one note.

- The forwarder printed the fixed line when the proxy closed an idle
  connection that never carried a request. An upstream end now counts
  only when the forwarder had sent the client's bytes; a socket error
  still always counts.
- Nothing tested that an error on one side ends the other. A new test
  streams from a proxy that keeps writing after a FIN, drops and resets
  120 connections mid-stream, and checks that the relay's and the
  forwarder's fd counts return to their baseline. With the error path
  cut it fails at 222 and 102 fds.
- A linger timer is now cleared when its socket closes by itself.
- spawnRelay's start timeout sends SIGKILL, so a relay wedged before
  "ready" cannot outlive it.
2026-11-21 16:22:32 -06:00
jason.woltje f904d7afda Keep a slow reader's tail, run the relay with no environment, and name a lost proxy
Review of 62468f963 found five things; this fixes them.

- Both copiers destroyed the other side on close, which cut a reply
  short for a client that half-closed and read slowly. A clean close now
  ends the other side and lets it flush; it is destroyed when its peer
  closes or 10 s after its last byte. An error still ends both at once.
- spawnRelay passed the caller's whole environment. The relay now runs
  with an empty one.
- The forwarder said nothing when the proxy died after the start check.
  A connection that ends before any reply byte, while the client still
  waits, now prints the fixed line once.
- stop() could wait forever on a wedged relay. It now sends SIGKILL
  after 2 s, and a stdin error cannot escape to the caller.
- New tests: half-closed slow reader through the relay and through the
  forwarder, empty environment, stop() on a stopped relay, the ppid poll
  alone, SIGINT and SIGHUP, and the proxy gone after the start check.
2026-11-21 16:22:32 -06:00
jason.woltje c27612cf41 Add the worker proxy relay and in-container forwarder, tested, not wired
Slice S2 of worker-proxy-simple. The host relay
(packages/processing-control/src/proxy-relay.mjs) listens on a 0600 Unix
socket in a caller-made 0700 directory and copies bytes to 127.0.0.1:<port>.
It refuses any other target, a socket path of 108 bytes or more, an unsafe
directory and an existing path. It exits on stdin EOF, on TERM, INT or HUP,
and when its parent changes.

The forwarder (adapters/pi/proxy-forward.mjs) listens on 127.0.0.1:<port>
and copies bytes to the socket. One start check through the socket must get
a reply byte; otherwise it prints one fixed line and exits 1.

Nothing calls either file yet except the tests. adapter.sh, compose.yaml,
the Containerfile and containers.mjs are unchanged.
2026-11-21 16:22:32 -06:00
jason.woltje 7264134899 Correct the final-stop notes from review, and name the cap and an unknown signal plainly
The header no longer claims stderr never carries an environment value: the
error line is pi's message as sent, and run-task redacts it. A comment says
why a worker fails a length stop with text while a managed session does
not. An unknown signal exits 1 rather than 128, and the over-cap line names
the 64 MiB cap. TOOLS.md notes that the wrapper is process 1 in the
container and passes SIGINT to pi.
2026-11-21 15:58:10 -06:00
jason.woltje 66d66eb736 Judge a pi worker run by its final stop reason, and print the cause of a failed run
In text print mode pi exits 1 on an error or aborted stop, but a length
stop exits 0 with nothing printed, so run-task recorded an empty success.
The worker entrypoint now sets MOSAIC_PI_FINAL_STOP=1 for headless pi.
adapter.sh then runs pi through final-stop.mjs, which uses --mode json,
keeps the last agent_end and fails the run on an error, aborted or length
stop with one plain line on stderr. A finished stop prints the answer as
text mode does. run-task also prints the cause line that show prints. The
reason stays exit-nonzero, with no new field. The mock adapter is unchanged.
2026-11-21 15:35:49 -06:00
jason.woltje baf5714443 Keep the Triage key state through a failed read, and stop a replaced decision draw
Re-check fix round. Triage settings hold the key state apart from the
read data, so a read that fails after a key change cannot let an older
read put back the old key state. The decision view checks that its draw
is still current after reading earlier versions, so a draw a newer one
replaced does not change the kept notice results.
2026-11-21 00:04:03 -06:00
jason.woltje 5f7adb7670 Keep a reload's route across a key change, and drop a result when its decision returns
Review fix round. A key change in Triage settings no longer drops a
Reload route read that was in flight. The read still draws its route
and revision, and keeps the newer key state. A decision notice drops
its kept result when the decision comes back into the inbox after a
draw saw it gone.
2026-11-20 23:52:05 -06:00
jason.woltje fc60809b80 Keep decision notice results, and drop older triage and workflow reads
The decision notice on an inbox decision kept its result only in the
page node, so the next poll redraw lost it, and a confirmed or reversed
decision left the inbox read with its notice. The result is now kept in
page memory per decision; a redraw keeps the result line and keeps both
buttons off while an answer is in flight.

Settings > Triage applied every route read. A read from an earlier visit
or a Reload route read that answered after a save or key change put back
the old route, revision or key state, and the next save was refused as
stale. Reads are now numbered as they start; one older than what the
page has drawn is dropped, and a save or key change takes a new number.

Workflows applied plain loads in the order they answered. Each
organization's loads are now numbered too, and an older load that
answers last, or fails last, is dropped. The SP17 write epochs are
unchanged.
2026-11-20 23:41:38 -06:00
jason.woltje 9b3cbe4296 Test that a hire receipt survives an overlapping packs reload
Two forced-order tests for the hire click. They fail without the packs state fix in 5a7e6765f.
2026-11-20 23:25:29 -06:00
jason.woltje 1e8a20074d Keep a newer autonomy read that lands during a save
Re-check of 7cba209dc (P3-1): a save can commit, another writer can
commit after it, and a poll can draw that newer read before the save
answers. The save then wrote its own older values into the state, and
the new redraw painted them over the newer form. It failed closed, but
the page showed older values.

A save that answers now writes its values into the state only when the
state still holds the revision it sent or the one it got back. Any other
revision was written after the save, so it stays. The redraw draws from
that state, so it never paints older values.

The older-read test waits for the draw as well as the read, since the
dashboard draws only after all its reads (re-check P3-2). A new test
forces the newer-writer order.
2026-11-20 22:48:55 -06:00
jason.woltje 2e78c7e593 Draw an autonomy save over a clean form drawn from an older read
Review of a4a035d17 (P3-1): a reload whose read was served before a save
committed could redraw a clean form while the save was in flight. The
save then wrote its revision to the form it started on, which the page
no longer drew. The drawn form showed the old levels next to the receipt,
and a second save before the next poll was refused as a revision
conflict.

A save that answers now draws the settings section again from the saved
state when the drawn form is a different, clean form. A form the owner is
editing stays, so it saves against its own revision and fails closed.
The live section lookup skips decision notices for the hold check too.

The older-read test now waits on the held read by the revision it
carries, so a poll in the window cannot move its count (P3-3). A new test
forces the P3-1 order and checks the drawn form before any reload.
2026-11-20 22:48:55 -06:00
jason.woltje fbfcd942be Keep an autonomy save or review receipt that overlaps a reload
The autonomy panel built a new state object on every load and copied the
status from the state it found when the load began. A save kept the old
object across its own request. So a save that answered while a poll or
an organization change was reading autonomy wrote its receipt and its new
revision to an object the page no longer drew, or the load copied the
empty status forward before the receipt existed. The next draw then
showed no receipt and the old revision, so the next save was refused as
a revision conflict. A trail review wrote its line only to the node it
started on, which a redraw replaces.

Each organization now keeps one state object. A load writes its read into
it when it ends. A load that began before the last applied read, or
before a save answered, is dropped. A save or review result is kept in
that state and written to the status line the page draws now.

Browser tests hold the fixture's calls to force each order of a save
against a reload, a review against a reload, and an older read that
answers after a newer one.
2026-11-20 22:48:55 -06:00
jason.woltje 5a7e6765f8 Keep a pack preview or receipt that overlaps a packs reload
The pack picker built a new state object on every load and copied the
preview and receipt from the state it found when the load began. A click
kept the old object across its own request. So a preview or a hire receipt
that answered while a poll or an apply's page reload was reading the packs
was written to an object the page no longer drew, or was copied away
before it existed. Under host load this dropped the project preview in the
SP6 pack picker test and the hire status in the pack hire test.

Each organization now keeps one state object. A load writes its read
fields into it when it ends, and a load that began before the last one
applied is dropped, so an older list never replaces a newer one.

One browser test holds the fixture's reads to force each order of the
overlap, a second forces an older packs read to answer last, and a third
holds a reload's packs read while an apply's receipt lands.
2026-11-20 20:33:47 -06:00
jason.woltje 69b3931c35 Resolve recheck follow-up cases 2026-11-20 19:43:53 -06:00
jason.woltje b7116be9c3 Address review of proxy administration follow-ups 2026-11-20 19:43:53 -06:00
jason.woltje 0f3135e36a Show unanswered organizations with offered models 2026-11-20 19:43:53 -06:00
jason.woltje b0dd4d3e2f Recognize organizations without a launch in model survey 2026-11-20 19:43:53 -06:00
jason.woltje e8c6c7e3a4 Test the 257 model limit boundary 2026-11-20 19:43:53 -06:00
jason.woltje fe671844ea Guard shared credential names with invalid account IDs 2026-11-20 19:43:53 -06:00
jason.woltje b3eaa8fd81 Preserve account removal success when refresh fails 2026-11-20 19:43:53 -06:00
jason.woltje 07baa87381 Clarify account removal ambiguity 2026-11-20 19:43:53 -06:00
jason.woltje 5e67fd9289 test(webui): wait for restart completion 2026-11-20 18:43:48 -06:00
jason.woltje ffc7db130f test(webui): enforce switch ordering in restart check 2026-11-20 18:43:48 -06:00
jason.woltje d6de22aaf8 test(webui): isolate browser waits and offline observation 2026-11-20 18:43:48 -06:00
jason.woltje f4134fff9a Fix account repair navigation and coverage 2026-11-20 17:03:22 -06:00
jason.woltje 122659a488 Make model setup account status truthful 2026-11-20 17:03:22 -06:00
jason.woltje 40afaaf6ad fix(harness): prove MCP seals with extension-backed loaders 2026-11-20 16:32:21 -06:00
jason.woltje 8042aa13ed test(cli): exercise plugin backstop with TMPDIR unset 2026-11-20 16:23:18 -06:00
jason.woltje 82aa0c0bda test(plugins): use a multi-segment temp path for redaction 2026-11-20 16:13:28 -06:00
jason.woltje 0430c7dc32 fix(release,task,auth): health-reason review round: the worker credential check reads the auth file path as compose does (a leading ~ is $HOME, a relative path is from the checkout, a named volume is undecided), treats COMPOSE_ENV_FILES and docker-compose.override.yaml|yml as undecided like .env, and leaves the credential not checked when this user can't read the auth file (EACCES, EPERM) instead of refusing; a key variable that is only spaces is no credential, as in pi; a compose.yaml that can't be read or parsed gives fixed words, never its path or text; release.sh's reason with no run id comes from the new scripts/health-reason.mjs (the last refusal line, prefix dropped, redacted, withheld when it holds an environment value) and says "gave no run id"; the fault-injection and recursion-guard comments are back; tests cover the tilde path, COMPOSE_ENV_FILES, an unreadable auth file, a blank key, a compose warning line in a failed run's stderr, and the redacted and withheld reason; docs/TOOLS.md names each undecided case (#1873, lane health-reason) 2026-11-20 15:31:59 -06:00
jason.woltje 1433686d50 feat(release,task,auth,runs,plugins): health-reason: a failed release health check prints its run id and one plain reason before release: refusing activate: health check failed: the cause line mosaic-task show now prints for a run that did not succeed (a runner reason in words, else one redacted line from the run's stderr.txt read through the new readRunText), or, when the runner refused before a run record, its own refusal line, redacted. A pi run checks before its run directory exists that the worker container compose starts holds a credential for config execution.provider: names and types from the PI_AUTH_FILE compose mounts and the key variables compose passes with a value, judged by pi-ai's own checkAuth on placeholders (packages/plugins/harness/pi/worker-credential.mjs), so no provider list is kept; with none it refuses exit 3 naming the provider, what the container has and what to do; when it can't decide (COMPOSE_FILE, an override or .env, an env_file, an unknown provider) it says so and the run goes ahead. auth.sh status adds one line for the default run's provider and credential. Tests in test-task, test-release, test-auth and runs.test; docs/TOOLS.md (#1873, lane health-reason) 2026-11-20 15:14:07 -06:00
jason.woltje d752e75359 test(model-proxy): models-offered re-check round 3: a model kept on while its account is signed out counts against the 256 limit, so a save past it is models-invalid before any in-use ask and writes nothing; a save keeps every model seen before, so one away at the save and back later is not New (#1873, lane models-offered) 2026-11-20 14:34:28 -06:00
jason.woltje dde3ee4140 fix(model-proxy,webui): models-offered re-check round: in use now counts every model or fallback.model key the organization file names (project agents, definitions; not launch.models' own upstream ids) and the root agents the organization takes (not the root assistant, not one that excludes it), resolved through the organization's model table or legacy mapping; an organization file or root agent file that cannot be read leaves that usage unknown (unanswered); the seed waits while any usage is unknown and keeps a model in use whose account is signed out on; a provider with no entry counts such a model as on, and a save keeps a model that is on but not present (not on the page) on; receipt lists (on, off, inUse, unanswered) stop at 32 with a count of the rest; the ask reads a user with no instance as an agent in its organization; history reads models.seed and models.set by name; tests cover each, plus the seed's revision recheck under the lock (#1873, lane models-offered) 2026-11-20 14:34:28 -06:00
jason.woltje acb228d56f fix(model-proxy,webui,bus,plugins): models-offered review round: an organization that rides the proxy but does not answer (bus down, org file unreadable, catalogue not ready) is listed as unanswered, and switching any model off then refuses models-in-use until confirmed, so a down organization is never treated as using nothing; a provider with no entry reads all present models on and a save above 256 that drops a model in use asks first; the read seeds the file only when a sending Console gets a same-origin request (seed:true), never on a cross-site, same-site or metadata-less read or on a read-only Console; the in-use body always carries usedBy and unanswered arrays and the ask names unanswered organizations; the receipt caps off, inUse and unanswered at 32 with a count of the rest; tests add a hard-linked and an extra-key file (damaged), Claude accounts of unknown, error, disabled and unavailable state keeping a key model claude-code only, the bus validator refusing an isNew other than absent or true, and a browser case for the unanswered ask (#1873, lane models-offered) 2026-11-20 14:34:28 -06:00
jason.woltje d01701a9ce feat(model-proxy,bus,plugins,webui): models-offered (A9): the instance owner switches proxy models on or off per provider; one state file <dataRoot>/state/models-offered.json ({version:1, providers:{<id>:{on,seen}}}, no harness, no key, written whole under the admin lock with a receipt); catalogue() takes offered: an off or unseen model is unavailable with model-off, an unseen one is New, a damaged file turns every model off and never falls back to all on, and harnesses never change; the bus reads the file at every catalogue read; Console GET /api/proxy/admin/models and POST /api/proxy/admin/models/set (instance owner only, revision, confirmInUse) refuse models-offered-conflict and models-in-use, the second naming each organization and instance that uses the model; the plugin has no rule for current models, so the first read seeds every present model on and seen; the Settings accounts part shows one list per provider with an in-place ask before a model in use is switched off (#1873, lane models-offered) 2026-11-20 14:34:27 -06:00
jason.woltje ccd85604c6 feat(bus,model-proxy,webui): account-remove: the instance owner removes one model proxy account. Bus verb proxy.account.remove {accountId, confirm:true} under the write lock and fence: a signed-in account's credential file is deleted with ManagementClient.deleteCredential (DELETE /v8/management/credentials {names:[name]}, as the panel PIN pins calls CLIProxyAPI 8.0.79's DeleteAuthFileV8), an API key leaves its key group and a group left with no key leaves whole, then the accounts are read again; a refused write is remove-write-refused, an account still listed is remove-unconfirmed; two listed accounts on one credential name or one id listed twice refuse before any write. The Console's owner-only GET /api/proxy/admin/accounts/impact names which models leave and which organization agents route to them (account-impact.mjs, unknown where the reads cannot decide); POST /api/proxy/admin/accounts/remove writes receipt account.remove with provider and account id only. Settings Model setup gets Remove with an in-place ask; a failed remove reads the list again and never says nothing was removed. providerUpstream in the cliproxy plugin maps a provider to its upstream (#1873, lane account-remove) 2026-11-20 13:27:54 -06:00
jason.woltje e035b6dea9 test(harness,conversation): pi-fit-tmp re-check 2 round: smoke.test removes the fixture directory conversation/tests/harness.mjs makes when it loads (after(cleanupAll)); with TMPDIR unset that directory now lands in /var/tmp, which a boot doesn't clear, and smoke was the one importer that never removed it; fixtureDir throws a plain line when the temp directory is under /tmp and /var/tmp is missing or under /tmp too, so a test fails on its fixture, not with the fit check's guard line 2026-11-20 12:51:40 -06:00
jason.woltje 735e828aa2 test(harness,plugins,conversation,cli): pi-fit-tmp re-check round: stand-in pis are built outside /tmp whatever the temp directory is, since the fit check now stays undecided for a pi under /tmp; the new fixtureDir(prefix) in pi-stand-in.mjs makes the directory under tmpdir() unless its real path is under /tmp, else under /var/tmp; every stand-in root in the harness, plugins, conversation and cli tests goes through it (harness-watch's piPackage gets its own root, its watch scratch stays under tmpdir() for the W1 redaction test); with TMPDIR unset pi-fit went 13/75 to 88/0 and claim 9/24 to 33/0; the guard's test gains a pi with only a root under /tmp; sessionCovers' comment sits above it, not above ended's 2026-11-20 12:37:08 -06:00
jason.woltje e698b7d785 fix(harness,seat,conversation): pi-fit-tmp review round: the fit check stays undecided for a pi mounted under /tmp or a node there ("can't be checked: the pi's package sits under the system temp directory, which a session's sandbox covers", or "node sits ..."), since a session's sandbox mounts its private /tmp last and covers it, and no run starts; mountOrder compares targets in plain form (a trailing slash or "." names the same directory), leaves an entry with no absolute target where it is, and its comment states the limit: no bind stays hidden under a later one, so only checkArgv uses it; the conversation controller's comment and README say why a node under /tmp still can't run; tests: the guard's three cases and /tmpx, mountOrder's plain-form, no-target and hidden-bind cases 2026-11-20 12:05:26 -06:00
jason.woltje 8ed3e251be test(harness): pi-fit-tmp: the /tmp test's link to its temp directory sits in the pi's package, which the sandbox mounts, so bwrap follows it; without the scratch under the real path that case fails 2026-11-20 11:44:14 -06:00
jason.woltje 784748347b fix(harness,seat): pi-fit-tmp: the fit check gives one verdict whatever the temp directory is; with the scratch under /tmp, the bind of the scratch's tmp/ over /tmp hid the scratch's own bind and every run failed before pi started ("can't be checked: --help failed"); checkArgv now orders its binds with the new mountOrder (a bind goes before any bind whose target lies under its own, nothing else moves), the scratch goes under the temp directory's real path, and a failed --help line says how the run ended (exit status N, ended by SIGNAL, timed out) with no path; tests: a check with its temp directory in one mkdtemp directory under /tmp, and through a link to it, gives the verdict a temp directory outside /tmp gives and leaves nothing; mountOrder's order; the --help line for an exit status and a signal 2026-11-20 11:42:54 -06:00
jason.woltje 9869200a03 fix(harness): pi-unpin S6 review round: tests now pin the checks no test held; a blocked read that skips the gate's hook and fails on its own (outside and each of the four spellings) is refused by the gate-reason check; a pi whose reads all fail is refused as not running the allowed tool; agent_end must carry the model's final text answer (stopReason and hasText in the turn marker), not just any object; a pi that finishes the script and exits non-zero is undecided; a pi that exits non-zero before any request is undecided with a line that says so; a pi that leaks a blocked read and then hangs is refused, not undecided; pi-stand-in gains hookSkipFail, readFails, exitCode, agentEndEmpty and a numbered modelHang; pi-session.test.mjs takes the expected pi from the pi module's own lookup; README matches (#1873, lane pi-unpin-s6) 2026-11-20 10:59:40 -06:00
jason.woltje be7066b725 feat(harness): pi-unpin S6: the fit check runs pi once against a scripted loopback model, so S0 lines 1, 2 and 4 and the system prompt are proved on the pi that runs; src/pi-fit-model.mjs is the model, first process in the sandbox, on the sandbox's own loopback with the network unshared, on a stand-in provider and model named fit-model; the script reads a file the gate allows, a file outside the project, the same file under each of the four read spellings pi 0.85.1 tries, and a file a second fit-only extension (src/pi-fit-throw.mjs) throws for; a pi that runs a blocked tool, swallows a handler's throw, drops the --system-prompt file or never emits agent_end is refused with harness-feature-missing's line, a run that times out or whose model gets no request is undecided, and the verdict is cached with the pass for the digest; helpers.mjs mockAnthropic shares the model's request handler; pi-session.test.mjs runs on the pi MOSAIC_PI_BIN names (#1873, lane pi-unpin-s6) 2026-11-20 10:59:40 -06:00
jason.woltje 4aa008e9fe test(cli,plugins,webui): harness-watch-w3 re-check: the plugin-root leak test skips when TMPDIR's parent is not writable instead of placing the root under TMPDIR, where it passed with the root left out of the backstop (N1); red and missing lines must be the page's danger colour, not only unlike muted (N2); a usable record after the first 256 is not checked yet (N3) (#1873, lane harness-watch-w3) 2026-11-20 10:26:35 -06:00
jason.woltje ca7f46e5d4 test(cli): harness-watch-w3 review P3-4: the plugin root in the write-time check test sits beside TMPDIR, not in it, so the test fails when store() leaves the plugin root out of the backstop (TMPDIR, an env value, caught a root under it anyway) (#1873, lane harness-watch-w3) 2026-11-20 10:26:35 -06:00
jason.woltje 256e80f26b fix(cli,plugins,webui): harness-watch-w3 review: readWatchView drops a file-only harness name that is an env value (P3-1), keeps at most 64 entries from the first 256 records with a Set for order (P3-2) and keeps only real times, so 2026-02-30 is none (P3-7); red and missing lines take the danger colour r3 gives can't start (P3-6); new tests: the default stat key reads the plugin root (P3-3), a record naming the plugin root is not written (P3-4), and after Save harnesses the lines stay or take the save answer's harnessWatch (P3-5) (#1873, lane harness-watch-w3) 2026-11-20 10:26:35 -06:00
jason.woltje 3225536adc feat(bus,cli,plugins,webui): harness-watch-w3: the bus plugins answer carries harnessWatch, one entry per harness (name, state, version, checkedAt, changedAt, failed, line) read from the host's <org dataRoot>/state/harness-watch.json by readWatchView (packages/plugins/src/harness-watch.mjs): read only, no link, FIFO or file over 1 MiB, and a missing, unreadable or wrong-shape file or record gives "not checked yet", never a throw or a guess; the read redacts again with the org and instance roots as known paths; Settings > Routing shows one line per harness under Save harnesses, red and missing in the warning colour, not checked yet never as fit, and one plain line when the answer has no list; the host's watch loads harness plugins from the instance root when there is one, as the launcher does, and keeps state under the org root; new tests: bus start starts the watch through deps.harnessWatch and --no-launch does not (#1873, lane harness-watch-w3) 2026-11-20 10:26:35 -06:00
jason.woltje 9287e2fac7 docs(agents): AGENTS.md "Version pin" becomes "Harness versions": no harness binary is pinned, package.json holds a floor range and the lock records what npm ci installs, the fit check and the host timer stand where the pin stood, and image tags derive from RELEASE and the lock (#1873, lane pi-unpin-s5) 2026-11-20 09:12:32 -06:00
jason.woltje 9d02ab4a69 test(release): pi-unpin S5 re-check round: gate-lane-image.test.mjs reads the expected tag from the copied lock instead of 0.85.1, so it follows the lock to a newer pi; pi-unpin-s5.test.mjs requires test-task.sh and test-auth.sh to stop when image-tag.mjs refuses; build.sh's header and docs/TOOLS.md say the tag comes from the lock, build.sh runs the fit step, and build.sh comes before test-task.sh since compose no longer builds a missing image (#1873, lane pi-unpin-s5) 2026-11-20 08:34:47 -06:00
jason.woltje 5210522e13 fix(release): pi-unpin S5 review round: compose.yaml loses its build section and sets pull_policy: never, so a compose run (hello.sh, agent.sh, the task runner) never builds or pulls the agent image past the fit step and refuses a missing one; build_image runs docker build -f Containerfile, refuses without MOSAIC_IMAGE_TAG, and with the gate's prepareImage is the only builder; test-task.sh and test-auth.sh stop when the tag can't be read; the gate test's order check no longer passes on a missing call; README and pi-fit-image.mjs say the fit check runs on the host's node, which can differ from the image's (#1873, lane pi-unpin-s5) 2026-11-20 08:34:47 -06:00
jason.woltje 8e9b05de8d feat(release): pi-unpin S5: package.json gives pi the range >=0.85.1 and the lock still resolves 0.85.1; the image tag is mosaic-poc-agent:<lock version>-r<RELEASE>, built by scripts/image-tag.mjs from package-lock.json for common.sh, mosaic-task.mjs, the gate's lane-image.mjs, test-task.sh and test-auth.sh, never from the package.json string; scripts/pi-fit-image.mjs runs the fit check on the pi the lock resolves, as npm ci installed it, before every image build (build_image in common.sh for build.sh, release.sh package and verify.sh, and the gate's prepareImage), so an unfit locked pi fails the build; it runs on the host because docker build's seccomp refuses bwrap; agent-host-dev.sh records pi --version for its banner and no longer refuses on version; docs match (#1873, lane pi-unpin-s5) 2026-11-20 08:34:47 -06:00
jason.woltje 0f07301607 test(plugins): harness-watch-w2 re-check: the two group-kill tests wait up to 15 s for the stand-in's background child to start, since the --help capture comes after the adapter runs and --version and missed 3 s under heavy load (#1873, lane harness-watch-w2) 2026-11-20 07:29:22 -06:00
jason.woltje a4d06b95c6 test(plugins): harness-watch-w2 re-check P3-3: the claude capture's stop test now has a background child that keeps the stdout pipe open, so it fails when a stop kills only the parent, and a new test does the same for pi's version capture on a stop (#1873, lane harness-watch-w2) 2026-11-20 07:29:22 -06:00
jason.woltje f2463ab36d fix(cli,plugins,harness): harness-watch-w2 review: pi-fit's two whole-result tests carry digest and undecided, and pi-binary's copy links packages/plugins/src (P2-1, P2-2); the claude fit rule counts a flag only from an option line read up to its description, so a flag a description only mentions is a miss (P2-3); on a cache miss preflight asks the bus host's watch for a pass at once (new src/watch-request.mjs: a stat pass now, a full pass on a changed key, one tick after a running pass, none after stop), and stays synchronous, so the launch that missed and launches before the first pass ends still go ahead (P2-4, the rest named in the README); the watch redacts word:/path, file:// and ~/ forms (P3-5); each claude capture kills its process group when it ends too and runs nothing on an aborted signal, the unobservable key-equal guard is dropped, pi inspect takes its check as an option, the host backstop leaves kept and source alone, and tests now fail without the group kill, the stop grace, the verdict cap, the early abort, the backstop's data root, home and process env needles and pi's undecided read (P3-6) (#1873, lane harness-watch-w2) 2026-11-20 07:29:22 -06:00
jason.woltje 7afe892e94 feat(cli,plugins,harness): harness-watch-w2: the bus host runs the harness watch on a timer (a full pass at start, a stat-key pass every 5 minutes, a full pass when the key changes or once a day, none while processing is halted or invalid and one on resume, never two at once, stop aborts the pass) and writes <org dataRoot>/state/harness-watch.json (temp then rename) and harness-watch.jsonl (one line per transition), checked for host paths and env values first; claude-code's fit rule: every flag adapter.sh passes (read from two recorded adapter runs) is in --help and --version answers, captured under unshare -r -n with pi-fit's fallback, a miss is fit fail with the flag as rule id, and preflight refuses harness-feature-missing on that kept verdict; pi-fit failures carry digest and undecided, pi's inspect reads them and its --version capture stops with the signal; the watch takes a signal and keeps flag rule ids (#1873, lane harness-watch-w2) 2026-11-20 07:29:22 -06:00
jason.woltje 4a3f501e13 fix(conversation): pi-unpin S4 sixth re-check: an engine node under /tmp no longer counts as one the fit check's sandbox shows, since pi-fit.mjs puts the fresh /tmp over it after it mounts that file, so a PATH with that node first refuses instead of letting the fit check run another node (R6-1); a lookup's problem line with a C1 control or a line or paragraph separator reads "the pi lookup gave no binary"; README and a test comment match (#1873, lane pi-unpin-s4) 2026-11-20 06:30:10 -06:00
jason.woltje c065034a6e fix(conversation): pi-unpin S4 fifth re-check: the own-copy rule for fit PATH entries covers /proc and /dev only; the sandbox's /tmp starts empty (pi-fit.mjs makes it fresh for each run), so a /tmp entry is judged like any directory the sandbox doesn't show, and S4h and a checkout under /tmp no longer refuse for it (R5-1); a lookup's problem line with a path or a control character in it reads "the pi lookup gave no binary"; S4h covers a /dev entry, a /tmp entry ahead of the engine's, and lookup lines (#1873, lane pi-unpin-s4) 2026-11-20 06:13:38 -06:00
jason.woltje 83250ef36b fix(conversation): pi-unpin S4 fourth re-check: under #!/usr/bin/env node, up to the first node every fit PATH entry must read the same in the sandbox as on the host: in plain form, so a .. after a link can't point elsewhere, and not in /proc, /dev or /tmp, which the sandbox makes its own, by its spelling or by the real path of the nearest part of it that exists; an entry or a node link the host can't check refuses, and entries after the first node are not judged (R4-1); the fit check gets the controller's label, which must be words like the lookup's, so no refusal line carries a hand-built label with a path or a newline (R4-2); S4h covers each case (#1873, lane pi-unpin-s4) 2026-11-20 05:53:43 -06:00
jason.woltje b432f9cd6f fix(conversation): pi-unpin S4 third re-check: under #!/usr/bin/env node the PATH walk follows what the fit check's sandbox shows (/usr, /etc, the host's /bin, /lib, /lib64 and /sbin, and the engine's node file at its path): a node the sandbox can't show, or reaches through a link it can't show, may not come before the first one it can, and that first one must be the engine's, so the fit check and the launch run the same node (R3-1); an empty or relative PATH entry refuses, since env reads it against the fit check's directory (R3-2); a label that isn't a plain string without a / reads "the pi" in a refusal line; S4h covers each case (#1873, lane pi-unpin-s4) 2026-11-20 05:36:07 -06:00
jason.woltje e0065b6a44 fix(conversation): pi-unpin S4 second re-check: under #!/usr/bin/env node the first node on the fit check's PATH (sessionEnv's: the repository's .bin, then this process's PATH) must resolve to the engine's own node and no node on it may lie inside the package the fit check mounts, or the start refuses engine-feature-missing, so the fit check and the launch run the same node (N1); the controller keeps a frozen plain copy of the lookup's answer, its own string data fields only, so a getter or a Proxy that answers differently later can't make the fit check read one package and the launch run another (N2); S4g covers a bin the lookup no longer takes (N3); a refusal line's label is the lookup's string label or "the pi"; tests in S4g and S4h (#1873, lane pi-unpin-s4) 2026-11-20 05:12:49 -06:00
jason.woltje 3f39ed3f11 fix(conversation): pi-unpin S4 re-check: a first line must end within the 256 bytes the kernel reads and carry no CR, and an absolute node in it must resolve to the engine's own node, so a package's own program named node or a long first line can't make the fit check run another program than the launch (P1); the lookup is run again on the answer's bin and must give the same root, dest, view and pkg, so a hand-built answer can't mount one package for the fit check and launch another (P2); S4g checks each field alone (P3); test S4h (#1873, lane pi-unpin-s4) 2026-11-20 05:12:49 -06:00
jason.woltje 28a5d38349 fix(conversation,discord): pi-unpin S4 review: the controller refuses engine-feature-missing for a pi whose first line doesn't run node (#!/usr/bin/env node, or an absolute path to a node), since the fit check runs the bin as a program and the launch runs its real file with node (F1); the lookup's answer must still hold together at each check, its bin and view resolving to the same file inside the package found, or it refuses unsealed-engine, so a package link moved since the lookup or a hand-built answer with another bin can't launch (F2); the Discord test helpers plant a pi package and name it in MOSAIC_PI_BIN, so no CLI test reads the host's pi (F3); the READMEs say what --repo and the old-claim line mean now (F4, F5); N24b proves the default lookup reads the controller's environment (F6); the stand-in pi gains a node form for the conversation fixtures; tests S4f, S4g and the S4e repeat (#1873, lane pi-unpin-s4) 2026-11-20 05:12:49 -06:00
jason.woltje 824b1213c1 feat(conversation,discord): pi-unpin S4: the controller launches the pi the harness lookup finds (binary.mjs locate, passed in as the pi option and defaulting to locate(process.env)) in place of pinRoot and PI_BIN; the seal compares the pre-argument to the real path of that bin, read again at each check; the version pin gives way to the fit check (pi-fit.mjs): a pi that lacks a feature or can't be checked refuses engine-feature-missing with the check's line, and the claim pins are engineVersion (the --version text) and enginePin (the fit digest), so changed bytes under a stopped claim still refuse engine-pin-mismatch; --no-mcp joins the seal when the pi's help lists it; a claim made under the old pin record refuses engine-pin-mismatch with a line saying to start a new claim; pi-pin.mjs is renamed pi-seal.mjs without the pin constants; the Discord CLI takes its pi from the same lookup; tests red on 29daa4c4b (#1873, lane pi-unpin-s4) 2026-11-20 05:12:49 -06:00
jason.woltje ca7388743b fix(adapters,harness): pi-skills re-check: the docs no longer call the fit check's last -p run a headless worker's argv, since task launches refuse skill directories and set MOSAIC_SKILLS empty (packages/processing-control containers.mjs) and only the interactive launch (scripts/agent.sh) sets it, in the TUI with --tools and --session-dir; the run is now named the nearest headless form of the adapter's --skill argv, and the Not run notes name the TUI launch's flags and --skill with no -e (R2-P2-1, R2-P3-1); the untested --system-prompt check added to that run is dropped, as at BASE (R2-P3-2) (#1873, lane pi-skills) 2026-11-20 04:30:23 -06:00
jason.woltje 59af86a5ee fix(harness): pi-skills review: the fit check's last -p run is now the adapter's headless worker argv with MOSAIC_SKILLS, an -e file and no tools (new export workerShape: --no-skills then two --skill directories, --no-tools, --no-session, the fit route's --provider and --model, --system-prompt, -p), and must list both given skills and no planted one, so a pi that leaks only with --skill and --provider is refused harness-feature-missing (P3-1); a test runs the adapter and holds its argv to workerShape; the run replaces the plain sealed -p --no-session run, so it still adds no run, and both Not run notes now name that plain shape, --skill beside --tools, --session-dir, -c, --thinking or --mode json, and one or three or more --skill directories (P2-1); tests red on ffe241789 (#1873, lane pi-skills) 2026-11-20 04:30:23 -06:00
jason.woltje d6d3271808 fix(adapters,harness): pi-skills: with MOSAIC_SKILLS the pi adapter keeps --no-skills and puts each --skill after it, since pi 0.85.1 adds a --skill directory to discovery (with --skill alone a scratch-HOME probe also loaded the user, settings, package and ~/.agents skills); the fit check's sealed -p run, the worker shape, now passes --no-skills with one --skill directory and must list that skill and no planted one, so a pi that lets --skill bring discovery back, or drops the --skill directory beside --no-skills, is refused harness-feature-missing, at no extra run; the stand-in pi loads --skill directories (skillLeak, ignoreSkill knobs); tests red on BASE for the argv order, real pi 0.85.1 through the adapter and both stand-in refusals; READMEs and the Not run note updated (#1873, lane pi-skills) 2026-11-20 04:30:23 -06:00
jason.woltje 4a7815d29e fix(plugins): harness-watch-w1 recheck: with no digest on either record the version stands in only when both records have one, so a timeout or a throw (no version) between two unknown passes is no transition (N1); README says the stored surfaces are redacted and so are for display and comparison, with fit rules run inside the module on the text the binary printed (N2), and that pi's "can't be checked" is read after its lookup label (#1873, lane harness-watch-w1) 2026-11-20 04:24:37 -06:00
jason.woltje af8c2f9777 fix(plugins): harness-watch-w1 review: a record carries lastDigest, the latest digest it knew, so a new binary after an unknown, red, missing, timeout or throw reads changed, and when neither record has a digest the version stands in, so red 2.0 to red 2.1 is a transition; the inspect answer is copied inside the try to own data fields only, so a getter, a Proxy or an inherited field is a failed inspect (unknown) and not a failed pass; redaction also covers realpaths, trailing slashes, the user name as a word (<user>) and any absolute path left (<path>), and the loader's refusal messages; a source or rule id equal to an env value is dropped; pi's "can't be checked" match reads only after its lookup label; claude-code skips relative PATH entries and hashes and runs the same resolved file; README updated (#1873, lane harness-watch-w1) 2026-11-20 04:12:24 -06:00
jason.woltje b156727c00 feat(plugins): harness watch W1: watchHarnesses() makes one pass over every harness the loader gives, with no name list, and calls the optional inspect({ env, scratch, signal }) or falls back to version() with fit unknown; it sets each record's state (fit, changed, red, unknown, missing) from the answer and the last record, gives a transition only when the state or the digest changes, and writes nothing; only found plus fit pass reads as fit, a fail is red only when definite, and a throw, a timeout or an answer outside the contract is unknown; source must be a lookup step's name, and every text field has the scratch directory, the home directory, env values and PATH entries replaced by names; pi's inspect reuses preflight's fit check and its cache (a "can't be checked" failure is unknown); claude-code's inspect gives the version, the sha256 and a --help surface from a scratch HOME, fit unknown; README documents the contract (#1873, lane harness-watch-w1) 2026-11-20 03:54:47 -06:00
jason.woltje 29daa4c4bf test(webui): settings-backend-a3 proof: the settings-review fixture's accounts answer carries the providers list, since with no list the Model setup page shows no sign-in button (F10 clicked it) (#1873, lane settings-backend-a3) 2026-11-20 02:21:46 -06:00
jason.woltje c8db3665cf fix(bus): settings-backend-a3 review: a provider list entry counts only with what its door needs, a sign-in host or a key base URL, so one missing it is refused provider-not-allowed before any proxy call (no sign-in opened at the proxy that the page cannot cancel), and a key group with no base URL is no provider's (P3-1) (#1873, lane settings-backend-a3) 2026-11-20 02:21:46 -06:00
jason.woltje f2c262cf40 feat(model-proxy,bus,webui): settings-backend A3: the provider list comes from the cliproxy plugin's PROVIDERS (id, label, sign-in kind, the sign-in host for a subscription, the base URL for the key provider); the management client's oauthStart, the bus proxy page's names, login allow list, sign-in host check and key allow list read it, with the same refusals (proxy-client-invalid, provider-not-allowed); the setup status and the accounts answer carry providers as id, label and sign-in kind only; the Model setup page builds the provider select and its help text from them, and with no list, an empty one or a failed read it shows no select and says the providers could not be read; an architecture test holds that no provider id, sign-in host or key base URL is written out in client.mjs, proxy-page.mjs or display.js (#1873, lane settings-backend-a3) 2026-11-20 02:21:45 -06:00
jason.woltje ebfbdb13a0 fix(webui): settings-s5c recheck: the status load says whether it painted the answer, and the result after an off says the list above is read again only then; a status read that answers 200 with null, an empty object or a page that is not JSON now says the list may be out of date, with Read again, instead of claiming the list is new (recheck P3); tests red on c47f9a3bd (#1873, lane settings-s5c) 2026-11-20 02:10:52 -06:00
jason.woltje 4d378fc38b fix(webui): settings-s5c review: an open Turn off question goes when a status read no longer lists that program or cannot list programs, so the buttons are not left disabled (P3-1); a refusal code named like an object property (constructor) shows as its code with the plain refused words, in failText and in the shared explain(), never as page source (P3-2); after an off the result says the list above was read again only when that read worked; when it failed it says the list may be out of date and gives Read again, which reads it and moves focus to the result (P3-3); invalid-request on an off says the program is not in the Console's installed list or that list could not be read (P3-4); the programs box keeps its own spacing inside the settings form (P3-5); tests red without each change (#1873, lane settings-s5c) 2026-11-20 02:10:52 -06:00
jason.woltje ab8633ede8 feat(webui): settings-s5c: Model setup, Advanced, lists the installed programs from the status read and which organizations have each one on, by display name, an organization whose list is null reading Not known in it, never off; Turn off for every organization asks first, naming the program, then POSTs programs/off, words each organization's outcome (bus-down never says stopped, failed:outcome-unknown says it may have changed, an unknown failed code shows the code unless it is shaped like a key), gives busy and a failed request a plain line, reads the status again and moves focus to the result; Technical details reads access health per organization when it opens: Ready only for ok with every key active and no reason, attention, unreadable, a missing or unknown health never Ready, no key id, agent or other count shown, a failed read with Read again; none of it shows or is read for a viewer who is not the instance owner; the old per-instance program read on Advanced is gone (#1873, lane settings-s5c) 2026-11-20 02:10:52 -06:00
jason.woltje 6ad11e6993 fix(pi): pi-unpin S3 re-check 10: unshare's outer layer is best effort: each fit check probes it once, alone and then with bwrap inside, and runs without it when unshare is missing, older than util-linux 2.36 or refused (Ubuntu 24.04 with apparmor_restrict_unprivileged_userns=1), with bwrap's argv the same either way and no refusal; only a failed bwrap or a missing setpriv refuses, each named; the sweep stops processes of this uid whose command line names a stale scratch dir before it removes it, scratch names carry the pid namespace and a sweep skips other namespaces' dirs, and a run or probe the check kills is swept at once; test 655 waits for pi.ready() before its fixture starts the launcher (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje e52a6b714f fix(pi): pi-unpin S3 re-check 9: a fit check dies with the process that runs it: its command line runs under setpriv's parent-death signal and a shell that checks its parent is still that process, then unshare puts bwrap as pid 1 of a fresh pid namespace with --kill-child, so a process bwrap leaves in its setup dies with the namespace, and bwrap gets --die-with-parent; each run's scratch directory carries its process's pid and start time and a check first removes those a process no longer running left; the fit route is zai glm-5.3-flash, with a test that goes red on a Claude model or Anthropic provider; the launcher's warm-up records a harness plugin it can't load once, by code only; test 655 waits for pi's ready() before its timed part, with its own timeout raised (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje 2d065b936e fix(pi): pi-unpin S3 re-check 8: the launcher starts pi's fit check in the background when it starts, through one harness load helper a launch shares, so the first launch joins the check in flight; nothing waits on it, a failed warm-up is dropped with no cached verdict and no unhandled rejection; a fit check that throws refuses as harness-feature-missing with no path in its line; tests for the launcher's rejected preflight and the Triage host's wait on ready(); the fit's provider and model move to the pi module's session-env.mjs (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje 098b3771c0 fix(pi): pi-unpin S3 re-check 7: the fit check runs its pi processes with async spawn, four at a time, and every caller awaits it, so the bus's event loop stays live (longest gap 22 ms against 6031 ms before) and two launches during one check share it; the launcher's call() awaits a module's promise and refuses its rejection as a throw, and preflight and version are awaited; the Triage host and route await the module's ready(); a run whose pipes outlive it settles a second after it exits or is killed; the fit's host env is the process's own through sessionEnv minus provider keys, its names are in the cache key, parity runs over a login shell's host and a unit's, and stand-ins that leak with TERM, TZ or LC_ALL unset are refused; bundle tests await harnessReady() (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje eac42663ba fix(pi): pi-unpin S3 re-check 6: the fit runs get a bus session's environment, built by the launcher's sessionEnv over ENV_ALLOW, the sandbox's sessionSetenv and the runner's adapterEnv with the pi module's harnessEnv (now in session-env.mjs), so no run sets PI_CODING_AGENT_DIR and the agent sources sit at $HOME/.pi/agent; a test fails when the fit env and a bus session's env differ by a name outside one listed exception; the drift test runs the adapter in that env over 98 argv sets, each tool count, -c case, thinking level and provider, and checks every variable the adapter reads; -c over a session dir with nothing for the project is named under Not run, with a gap test (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje b41bc00b26 fix(pi): pi-unpin S3 re-check 5: five -p runs in a bus session's shape pass every pair of the adapter's session branches (tool list or --no-tools, -c over a planted session, --thinking, --mode json), a -c run must resume the planted session, each loads two -e, the -e extension's command list shows skills and prompt templates in -p, the drift test drives the adapter through all sixteen branch sets and fails on a pair no run passes, and a test holds the --no-mcp run on its own (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje 749d6d98f5 fix(pi): pi-unpin S3 re-check 4: one sealed run in a bus session's shape, built by sessionShape, which a test holds to the adapter's argv; a one-out run must show only its own kind; the README and header list what a session passes that no fit run does (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje fa5ce3ae65 fix(pi): pi-unpin S3 re-check 3: the control passes --no-tools and --no-themes, one sealed run per loader flag leaves only that flag out and its kind must show, every run gets a fresh scratch directory, and the README says what argv the check runs (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje d009ccaec4 fix(pi): pi-unpin S3 re-check 2: the control passes the sealed runs' -e and --system-prompt so the runs differ only by the seal flags, a project source no longer shows a kind, a run over 4 MB says so, and --version gets 3 s (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje 2613fb1c4c fix(pi): pi-unpin S3 re-check: the control counts a kind only through a source the sealed runs can show, so a SYSTEM.md no longer stands in for the project APPEND_SYSTEM.md or an AGENTS.md (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje df26085303 fix(pi): pi-unpin S3 review: the control must show each planted kind and answer get_commands, the -e extension reads the system prompt so context and trusted project files are checked, settings and package sources are planted, help flags come from the option column, missing -e and the seal run in -p too, and the digest covers npm's lock files (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje 7c1b87d154 fix(pi): pi-unpin S2 review: a found pi must be the bin its package's package.json names, and the fixtures name it (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje 8908df113e feat(pi): pi-unpin S3: every start runs a sandboxed, offline fit check on the pi it found and refuses harness-feature-missing when a flag is missing, a missing -e starts, or a planted source loads with the seal flags on (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje b8354e4304 test(pi): pi-unpin S2 follow-up: the adapter tests point MOSAIC_PI_BIN at their stand-in, and the sandbox test reads the new mount label (#1873) 2026-11-20 01:17:33 -06:00
jason.woltje f62b1b9518 feat(pi): pi-unpin S2: the session runs the pi the module finds: MOSAIC_PI_BIN, else PATH, else the repository's copy, mounted by its package directory (#1873)
binary.mjs locate() takes MOSAIC_PI_BIN (absolute and executable), else the
first pi on PATH past the repository's node_modules/.bin, else the
repository's copy. The binary must belong to the pi package. mounts() and
installRoots() hold that package directory; the repository's copy keeps the
whole node_modules mount, since its packages are hoisted beside pi. env()
sets MOSAIC_PI_BIN to the binary's path inside the session, and the adapter
and the model-stream branch run it. sessionEnv passes MOSAIC_PI_BIN, and
Triage's SPAWN_PATH_ALLOW takes it with every host check kept.
harness-binary-missing keeps its code; with no pi its message is "no pi
found: set MOSAIC_PI_BIN, put pi on PATH, or run npm ci". A refusal names
the variable, never a host path.
2026-11-20 01:17:33 -06:00
jason.woltje 7355b233a8 feat(pi): pi-unpin S1: pi's labels come from the binary the module finds, not a pin (#1873)
version() runs `<bin> --version` with a 5 s timeout and a PATH that finds node, and returns
null when that fails or prints something other than one short printable line. The compaction
count records the found package's own package.json version, else null. Adapter and module
comments say "read from 0.85.1" instead of "pinned". The lookup still honours only the
repository's copy; S2 widens it.
2026-11-20 01:17:32 -06:00
jason.woltje 88d08c8437 fix(model-proxy,webui): settings-backend-a2 review: turning a program off counts the root agents each organization takes (not excluded, not the root assistant, as materializeCopies copies them), so a program only a root agent uses stays on (in-use), and a root agent file that can't be read, or no root reader, writes nothing where the program is on (unreadable); the Console passes loadRootAgents and the root file is read once per call (P1-1); tests for a revision that is not 64 hex, routes from another revision, routes with no instances and a lock-pinned harness listed as installed (P3-1); the busy test no longer hangs when the name check is gone (P3-2); an organization's harness list with anything but plugin names reads on: null (P3-3) (#1873, lane settings-backend-a2) 2026-11-20 00:01:05 -06:00
jason.woltje 4377941bd4 feat(model-proxy,webui): settings-backend A2: the proxy status lists the harness programs installed on this instance (from the plugin tree) and which ones each organization has on, with no organization needed; POST /api/proxy/admin/programs/off { name } turns one program off in every current organization through plugins.set after a fresh read, skipping an organization where a role route, a fallback or a project agent in its file uses it (in-use) or where it is the last one (last-program), writing nothing for an organization that is down or unreadable, reporting a revision change as failed:revision-conflict without retrying, one call at a time (proxy-admin-busy), with one receipt of name and outcome ids; GET /api/proxy/admin/access gives each organization a health and its stack-managed key ids from proxy.keys, never a key, never the usage queue, and a proxy-uncertain answer reads attention; both routes are the instance owner's only; the Console's riders carry the organization file for the in-use check (#1873, lane settings-backend-a2) 2026-11-20 00:01:05 -06:00
jason.woltje de39750ebb fix(webui): settings-s5b recheck: after an empty history page, folding the list keeps Show older, since the line still names it (N1); a test holds that a successful Read again moves focus to Show older (N2); tests red on 3b17326b7 (#1873, lane settings-s5b) 2026-11-19 23:30:29 -06:00
jason.woltje 3b17326b74 fix(webui): settings-s5b review: Show older shows whenever there is an older page and nothing read is folded away, and what it reads stays open; Show all only when entries are folded (P2-1); an older page not further back than the one asked for is outcome unknown (P3-1); when the pressed history button goes, focus lands on Read again, Show older, Show fewer, then the heading (P3-2); a service that is not installed has no lines, said plainly, not as a failed read (P3-3); an unknown history action reads Other setup action, even one named like an object property (P3-4); a hidden user reads "someone whose name Mosaic hid" (P3-5); a test holds a stale history answer (P3-6); the service key is the last row of the state list and fails in the same banner style with Read again (P3-7); the setup-stops test now refuses a real unit start; tests red on d03ae494e (#1873, lane settings-s5b) 2026-11-19 23:17:08 -06:00
jason.woltje d03ae494e8 fix(webui): settings-s5b: in Technical details the service key and the history sit right after Release and Service, as models-advanced shows them, with the service's last lines after the history; history entries are not numbered (#1873, lane settings-s5b) 2026-11-19 22:59:03 -06:00
jason.woltje e1b56d9ea7 feat(webui): settings-s5b: Model setup's Technical details shows when the service key was made (GET /api/proxy/admin/service-key, never the key, with renewal not built yet), the setup history a page at a time (GET /api/proxy/admin/history, withheld rows plain, an empty page with a next page still offers Show older) and the service's last lines (GET /api/proxy/admin/log, as the answer gives them); a failed step 1 opens the lines; each read waits until its box opens, a failed read says so with Read again, and none of it is read or shown for a non-owner; the raw receipts list is gone; leaving the Models page for any page drops its focus request; tests red on e6f6f5445 (#1873, lane settings-s5b) 2026-11-19 22:53:50 -06:00
jason.woltje e6f6f5445b fix(webui): settings-s5 recheck: coming back to an organization's Models page reads the model service again, so the not set up line goes once the owner has set it up (N1); a switch left while it writes no longer moves focus to its outcome line on the next visit, and a render that paints no outcome line drops the focus request (N2); tests red on da123f6dd (#1873, lane settings-s5) 2026-11-19 21:48:50 -06:00
jason.woltje 55f9652f6a fix(webui): settings-s5 review P2-1: while an organization needs a restart its Models page leaves out Stop using your models, as the org-models-restart mockup does, since buttons here have no disabled look; the shots add the model service not set up state (#1873, lane settings-s5) 2026-11-19 21:48:50 -06:00
jason.woltje 9e434c0e9c fix(webui): settings-s5 review: on an organization's Models page Stop using your models waits while the organization needs a restart (P2-1); the header says the models come from your accounts only when the plugins read says the organization uses them, and from its own organization file when not (P2-2); a choice not in force says when the model service isn't set up or isn't answering, with a link to Model setup, from the owner's status read (P2-3); after a switch focus moves to the outcome line (P2-4); Use your models shows Saving while its write is out (P3-1); a failed read before the write says "Not changed. Nothing was written." with Read again (P3-2); a Console with no organizations says "this business" mid-sentence (P3-3); the choice sits in the form column at 1440 (P3-4); tests red on e4bb2be14 (#1873, lane settings-s5) 2026-11-19 21:48:50 -06:00
jason.woltje e732ae7450 test(webui): settings-s5: shots of Acme Studio's models (not using your models, after Use your models, needs a restart), the account reasons, step 3 can't start, Technical details and Messages at 390 and 1440, light and dark, measured for sideways scroll and focus rings; "No accounts yet." on Model setup reads as one line instead of squeezing into the name column, with a test red without the shell.css rule (#1873, lane settings-s5) 2026-11-19 21:48:50 -06:00
jason.woltje 66255c9bf2 feat(webui): settings-s5: an organization's Models page offers its one choice, "Where <Org> gets its models": Use your models (the cliproxy model-provider module, written through plugins.set after a fresh read, refused when the page is stale or a route draft is open) or Use its own model service, dashed, disabled and badged Coming later; the page says from the plugins read which one is in force and what a switch changes, Stop using your models is the way back, a refusal keeps the read's choice, a failed read offers Read again, and while the organization needs a restart its banner's Restart is the one primary and the choice waits; the sidebar marks read the status again after a switch; tests for slice 5 and the slice 4 items, red on 524132b42 except the no-Details control case (#1873, lane settings-s5) 2026-11-19 21:48:50 -06:00
jason.woltje cecc901854 fix(webui): settings-s5: a failed turn in Messages whose body carries its own "\nDetails: " part shows its cause and "Details:" once, split on the first "\nDetails: " as chat.js does; #/settings/members and #/orgs/<id>/settings/members land on that organization's Agents page at #/settings/agents?org=<id> (#1873, lane settings-s5) 2026-11-19 21:48:50 -06:00
jason.woltje 81ca8399a2 fix(webui): settings-s5: Model setup names each organization by its display name in the Move confirm, its result, the not-moved line and the skipped lines, and a stand-in organization's line says "your models" in step 3's words; before the first account step 3 names an organization the sidebar marks "can't start", with a Can't start badge, and stays closed; Technical details and Advanced show the timeouts init chose (firstOutputSeconds, rotateGraceMinutes, "the default" when proxy.json leaves them unset) from the status read; MosaicModelDots gains read() so another page can ask the marks to read the status again (#1873, lane settings-s5) 2026-11-19 21:48:50 -06:00
jason.woltje 37c3854849 fix(model-proxy): settings-backend A1 review P3-1 and P3-3: a short Mosaic key glued to a word (xmso-..., _msk-...) is redacted by one added line after the copied key-prefix line, which the pin test now expects; a history entry whose redacted values, or keys and values, read together still hold an environment secret split across them is answered as { withheld: true }; comments name the split-key journal limit (P3-2) and the unmarked non-object answer (P3-9) (#1873, lane settings-backend-a1) 2026-11-19 21:09:58 -06:00
jason.woltje db2dc96aa2 feat(model-proxy): settings-backend A1: three instance-owner reads under /api/proxy/admin/ for Model setup: the model service's last log lines with keys and secrets removed (log), the service key's made-at time from proxy.json's mtime with no renewal yet (service-key), and setup history a page at a time (history); account writes through /api/proxy/* and an organization's proxy routes now join that history by label only; redaction is chat-fail's list with the mso-, mst- and msp- prefixes added, pinned to failure-notice.mjs (#1873, lane settings-backend-a1) 2026-11-19 21:09:58 -06:00
jason.woltje f666590d54 fix(proxy-scope): slice 4 review round 1: a save runs ensure only when it creates the route, so after Turn Triage off only Repair access brings a key back; the key walk stops at a bus whose outcome is unknown and the whole walk has one 30 s deadline; the status reads the same riding list the walk asks; a repeated sweep refusal is logged when it appears or changes and then once an hour; Off asks first and each key action has its own plain line; the host send-path test asserts the stop before it waits (#1873, lane proxy-scope-s4) 2026-11-19 20:41:55 -06:00
jason.woltje 2f20346719 feat(proxy-scope): slice 4: the bus issues, rotates and retires the Triage key through proxy.triage.key (op ensure or revoke), for the instance owner's path-bound proof only; a riding bus sweeps a due rotation or retirement under the instance proxy lock, re-reads the file, never issues the first key and does the work once across buses; the Console asks one riding organization's bus on the first save, Repair access and Off, shows triage-no-riding-bus and rotation overdue, and stops a session at its key's retireAt with triage-key-rotated; answers, events and receipts carry key ids only (#1873, lane proxy-scope-s4) 2026-11-19 20:41:55 -06:00
jason.woltje 524132b42b test(webui): settings-s4 review F1: each organization pick waits for the picker and changes it in one step, since a route change repaints the select; the final gate caught the select missing for a moment between a wait and the pick; the test still fails with the bus.js and display.js of e717824c5 (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje ae53b1988a feat(webui): settings-s4: an account row on Model setup shows why the account is not working, from the accounts read's reason field (chat-fail F3, a7362a8fc and later), as text beside its state; with no reason field, or an empty one, nothing shows; the test is red with the display.js of 12e99db49 (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje 940ac53b65 test(webui): settings-s4 review: the picker from Model setup, Appearance and Organizations lands on #/settings/routing?org= (F1); the can't start mark and the step 3 words and Ride all line agree (F2); no paused mark with only a fence to clear (N8); every control under Advanced shows a focus ring before setup, and the fixture service reads healthy (N9); F1, F2 and N8 fail with the display.js and bus.js of e717824c5 (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje 799df9cb9f fix(webui): settings-s4 review F2 and N8: an organization the sidebar marks "can't start" (stand-in models, bus stopped) is named on Model setup step 3 by its display name in the same words, with the fix (move it to your models, then start its bus), and its Ride all line and Move button use the display name; the paused mark needs the service itself down (not running or not listening), so a fence left to clear no longer marks organizations paused (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje 23964b1224 fix(webui): settings-s4 review F1: the organization picker on a page that is not an organization's (Model setup, Appearance, Organizations) opens that organization's Routing page at #/settings/routing?org=<id>, so the address base no longer changes to #/orgs/<id>/ (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje d692ca9b81 fix(webui): settings-s4 review F3: Model setup drops what design section 10 removes: "Use my own key" and its key fields (the service key is always made on this machine, so init always sends generate), the key rotation grace field and its settings row, the Client keys table with its refresh button and the keys read, and the "Accounts and client keys loaded" line; the setup cleanup no longer clears the removed key fields; the Advanced help says Claude Code, not "the Claude harness" (N5); the tests that pinned the removed parts read the new page (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje 57e86d9867 test(webui): settings-s4: tests that pinned the old addresses read the slice 4 ones: the dashboard's Model setup row and Triage's fix link open #/settings/model-setup, and the single-pane switch tests go to #/settings/<page>?org=beta (their #/orgs/acme/... starts still pass through the old-address redirect) (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje d502a75e57 test(webui): settings-s4: the Business settings test reads the slice 4 nav (You, Your models, This instance, This business with Business last) in place of the old Your browser and This instance list; Business is still in Settings and not in the sidebar, and still visible in the phone disclosure (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje 15801ccb07 fix(webui): settings-s4: Model setup shows one primary button: a submit there reads as a plain button unless display.js marks it primary, so Finish sign-in and Check and add no longer look primary beside the card's own; the sign-in row wraps with a gap at 390; the test counts primaries by their drawn fill, and checks one primary while signing in and with the key form open (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje 50ca7f0341 fix(webui): settings-s4 shots round: a progress line reads what happened at each step (done, under way, not started, or "did not finish" where the run stopped), so a failed step no longer reads "Starting the service…"; an account's health badge keeps its own width on a phone; the model search box has a placeholder; the slice 4 test file takes shots at 390 and 1440, light and dark, with MOSAIC_UX_SHOTS set, and checks sideways scroll, select fit and a visible focus ring on every control it tabs to (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje b9b3e56821 test(webui): settings-s4: nav groups, the picker and ?org= pages, the old addresses, the phone section button with no sideways scroll, the three sidebar marks, the one-press chain in order, a failed step with Try again and last lines, ?step=, account rows and the provider-picked add form with sign-in cancel and paste-back, Models offered read-only with search, and the done line with a removed organization; all 11 fail on 6c5ec23b4 (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje 32b56b8351 feat(webui): settings-s4 sidebar marks: an organization whose models need a restart reads "needs restart" (◐), one on stand-in models whose bus can't start reads "can't start" (▲), and one riding a model service that is set up but not running reads "paused" (◐); restart and paused show only on a running bus; the dashboard's Model setup card links #/settings/model-setup; an organization's Settings page marks it in the sidebar (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje 78d1ef7705 feat(webui): settings-s4 Model setup: four step cards (Model service, Accounts and models, Organizations, Triage) with at most one primary button on the page; one press runs download, set up, install and start in order, each step once, reading the status after each, with a progress list; a step that fails stops with Try again and opens "The service's last lines"; done cards fold to their line, ?step=<card> opens one; accounts as rows with kind and a health badge, a provider select that offers "Sign in to <provider>" or "Check and add" for an API key, a cancellable sign-in with the paste-back form, and "Removing an account is not built yet."; Models offered read-only per provider with a count, a search box past eight and "Coming later"; Advanced and Technical details folded; the Ride all done line does not count an organization that turned the model service off; MosaicModelDots reads the setup status for the sidebar marks (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje af1bbf7859 feat(webui): settings-s4 nav and addresses: Settings nav in four groups (You; Your models with Model setup and, for the instance owner, Triage; All organizations; This organization with an organization picker above Models, Agents, Launch limits, Token use, Assistant and Organization file); an organization's pages live at #/settings/<page>?org=<id> and the old #/orgs/<id>/settings/<page> form lands there; #/settings/model-proxy opens Model setup at #/settings/model-setup; the phone section button reads "<Org> · <Page>"; the breadcrumb names the organization on its Settings pages; no Members row (#1873, lane settings-s4) 2026-11-19 18:49:24 -06:00
jason.woltje a7362a8fc5 test(webui): tasks-board: the fixture task is due in 2099, so the card still reads due, not overdue, after 20 Nov 2026 passes; the expected label carries the year (#1873, lane chat-fail) 2026-11-19 18:20:43 -06:00
jason.woltje 5a07f9e1f9 test(bus): chat-fail pickup-launch: the fixture runs its Claude model on claude-code, since a legacy Claude entry counts as OAuth and runs there only; the host launcher's env carries a stand-in session login, because claude-code's preflight needs one and this launcher boots with no proxy, and a stand-in adapter path skips only the binary check, so each launch still stops at the model gate with model-provider-unavailable and no assertion changed (#1873, lane chat-fail) 2026-11-19 17:08:06 -06:00
jason.woltje 07eca4a670 fix(chat-fail): review round 3: the link, email and mixed-run redaction patterns start only where a run without spaces starts, so redacting the whole kept stdout stays linear (a 128000-character run took 10 to 15 s, now a few ms) and matches the same text; a long text that redacts below the read limit is not cut and keeps its first word, and a cause that is only [redacted] is dropped; the proxy page reads relogin, plural and past forms of an expired sign-in as a sign-in (#1873, lane chat-fail) 2026-11-19 16:36:16 -06:00
jason.woltje fc9e4dfca8 fix(chat-fail): review round 2: the F1 cause is redacted over the whole text before the tail cut, so a cut that lands between a key word and its value no longer shows the value; the account sign-in words in the proxy page match whole words only ("assigning" and "logging" no longer read as a sign-in) and a prefixed or "is expired" token counts (#1873, lane chat-fail) 2026-11-19 16:36:16 -06:00
jason.woltje 10a35ce65a fix(chat-fail): review round 1: the F1 cause decodes %XX, HTML references and NFKC forms before the secret and redaction checks, redacts the value after a quoted or space-separated key word and Windows and UNC paths, and is read from the tail of the output, with the cause on the retry log line too; a launch-failed start posts only when the same message is refused twice, project-pending and processing-halted are waits, and a refused post is tried again and logged once; the proxy reason refuses an impossible date, checks quota_blocked before the message words and no longer reads a bare "expired" as a sign-in (#1873, lane chat-fail) 2026-11-19 16:36:16 -06:00
jason.woltje 3b9cac5452 fix(bus): chat-fail F3: an account the proxy reports as unavailable or in error carries one plain reason with the next step, built only from the fields of the existing account read (requires_sign_in, refresh_error, quota_blocked, next_retry_after, and status_message matched against fixed patterns and never shown); with nothing more it says the model service gave no reason. No new proxy call; working, turned-off and unknown rows keep their exact fields (#1873, lane chat-fail) 2026-11-19 16:36:16 -06:00
jason.woltje bc6d3565b0 fix(bus): chat-fail F2: a refused thread start and a proxy-stall stop each put one plain line in the chat, with the cause and next step, once per unanswered message and not once per retry; a start that only waits (a free place, a run still ending, a held lock, a lost race) puts none. The line comes from the launcher through a trusted launch op, not a socket verb, from the thread's instance with from_run bus, so no schema change and no new refusal code (#1873, lane chat-fail) 2026-11-19 16:36:16 -06:00
jason.woltje 72ea7272ec fix(harness): chat-fail F1: the turn-failure code moves into the mounted harness failure-notice.mjs, so a sandboxed runner loads it; the separate file was outside RUNNER_FILES and every sandboxed run died with ERR_MODULE_NOT_FOUND (#1873, lane chat-fail) 2026-11-19 16:36:16 -06:00
jason.woltje 27a359590e fix(harness): chat-fail F1: a failed turn's chat line carries the harness's error as a plain cause and next step, cleaned, redacted and capped, with the technical reason under it (#1873, lane chat-fail) 2026-11-19 16:36:16 -06:00
jason.woltje 6c5ec23b47 test(model-proxy): settings-p3 ride-fence: the launch fixture runs its Claude model on claude-code, since a legacy Claude entry counts as OAuth and runs there only; a stand-in adapter path skips only the binary check, so the before launch still reaches the limit checks and the fenced launch after Ride all is still refused with proxy-uncertain (#1873, lane settings-p3) 2026-11-19 15:44:31 -06:00
jason.woltje dd0a8612c8 fix(webui, model-proxy): settings-p3 review P2-1, P3-1 to P3-3: the done line counts an organization that can still ride, so an owner who can't send is not told every organization can see the models; an organization that took cliproxy during a Ride all run reads as already using it, not refused; the stopped page's refusal repaint is tested before the page renders again; a stand-in organization with no bus status (the single-business Console) reads that moving is not available there, with no Move button, from the status summary as well as after a run (#1873, lane settings-p3) 2026-11-19 15:16:59 -06:00
jason.woltje 9276277970 test(webui): settings-p3 shots: the Model setup shot shows the Ride all box below the restart list, with one organization that can ride, one skipped and one on stand-in models; a full-page shot grows the window to the page so the fixed footer covers nothing (#1873, lane settings-p3) 2026-11-19 15:04:56 -06:00
jason.woltje 517a3fe467 fix(webui): settings-p3 P3-1 to P3-11, rebase re-check 1 to 4, Routing help: Model setup says what to do for a held init lock or a partial init, with the README path under Technical details, and both lines are tested; the triage-drawer fixture case says its pair is synthetic; a model-key collision names a step; a fixture refusal leaves Restart on the stopped page but not primary; a ready catalogue the bus picked up late keeps its organization on Model setup's restart list with a Restart, holds the next step, and gets a Restart beside its notice on Routing; the README names the Model setup page; the NO_GUESS pin counts both uses; a reason that is not a stop reads "Last check:"; the why line sits under the headline; orgDownReason tells every waiting caller, each with its own name; the done line does not say every organization while the Ride all box lists one; Ride all is the next step only when the owner can send; the step button reads "Go to Ride all"; Ride all text says model service; the single-business fixture move says nothing about a bus it can't see; the Routing help names no harness (#1873, lane settings-p3) 2026-11-19 15:04:56 -06:00
jason.woltje 149712816b fix(model-proxy): settings-p3 R1, R2: Ride all reads each organization's file and change log again before its add, not only the receipts, so an add and remove saved on its Settings page during a run is not put back; a list that can't be read again, a change log that can't be read or an organization no longer listed skips the add as unreadable; with no bus status (the single-business Console) a fixture move answers not-available instead of bus-running, and whether its bus runs is unknown (#1873, lane settings-p3) 2026-11-19 15:04:56 -06:00
jason.woltje 4f63176676 test(triage): review round 5: a host bundle file replaced with a FIFO is refused before the host reads it (#1873, lane harness-matrix) 2026-11-19 14:43:26 -06:00
jason.woltje 1e0aba8b37 fix(triage): review round 4: the host refuses a pi command value (a string that starts with "!") in any file the module wrote, checks each host bundle file after env() whether or not the scan reaches it, and records ctime for the rest of the run directory (#1873, lane harness-matrix) 2026-11-19 14:43:26 -06:00
jason.woltje 680bb90af3 fix(triage): review round 3: the host checks every URL in a file the module wrote under bundle/, home/, work/ or tmp/, as written or as a URL parser reads it; refuses a backslash in a file JSON.parse refuses; refuses a change to its own bundle files or the rest of the run directory; and a refused start removes work/ and tmp/ too (#1873, lane harness-matrix) 2026-11-19 14:43:26 -06:00
jason.woltje 6a24cf743d test(triage): a declared harness file with a second hard link is refused (#1873, lane harness-matrix) 2026-11-19 14:43:26 -06:00
jason.woltje 06ee172a9d fix(triage): review round 2: the host reads every file under the run's bundle/, home/, work/ and tmp/ after bundle() and env(), refuses a link or a key there, and reads a JSON file decoded too, so an escaped key or URL is seen as the harness sees it (#1873, lane harness-matrix) 2026-11-19 14:43:26 -06:00
jason.woltje 66676804eb test(triage): write() refuses a bundle file that holds the key before the file exists, so the key never rests on disk until the scan (#1873, lane harness-matrix) 2026-11-19 14:43:26 -06:00
jason.woltje a3ad6578c4 fix(harness): review round: a legacy model entry (a provider or route name, no module) carries no account data, so a Claude-named model on it is treated as OAuth and held to claude-code; the Triage host refuses a key in any bundle/ file and any URL but the proxy's in a harness file; the models.json comment says what route() can and cannot see (#1873, lane harness-matrix) 2026-11-19 14:43:26 -06:00
jason.woltje 77a951cfe7 fix(harness): the harness rule keys on the account kind: a Claude model from an API key runs on claude-code and pi; an OAuth Claude model, or a group model whose name an OAuth Claude account offers, runs on claude-code only (#1873, lane harness-matrix)
Jason 2026-11-19: Claude models through an API key may run on any harness; OAuth Claude models only
on claude-code. The name check is gone. The catalogue reads every Claude OAuth account's model names,
whatever the account's state, and holds a group model with one of those names to claude-code, since
the proxy routes by name across every account. route() has no account data, so it holds upstream
claude only; admission and Triage hold the rest through the catalogue.
2026-11-19 14:43:26 -06:00
jason.woltje cd58ac23cd feat(harness): an OAuth Claude model runs on claude-code only; Triage runs on claude-code or pi through the proxy (#1873, lane harness-matrix)
cliproxy holds a model to claude-code when its upstream is claude, when an OAuth Claude account
offers the same name, or when the name contains "claude"; every other proxy model runs on
claude-code and pi. The catalogue, the launch (sessionModel and the models.json list) and the
provider route apply the one rule and refuse with model-harness-incompatible, so a stored
Claude-on-pi route is refused, never rewritten. The other route's pi provider is mosaic-proxy-other.

Triage: the route asks the provider for the transport on the stored harness and holds the
harness's env() to one contract (the key, unchanged, under exactly one of ANTHROPIC_AUTH_TOKEN or
ANTHROPIC_API_KEY; the URL only as ANTHROPIC_BASE_URL), with no list of harness names. The host
gives the module the real proxy and transport, checks each file it writes under the session's HOME
(regular, one link, no key) and binds it read-only after the run directory, and refuses a route env
with no key name or two. PATH starts with node_modules/.bin, as a bus session's does.

Tests: claude-code runs a z.ai and a codex proxy model against a stand-in (real claude binary);
pi answers Triage through a stand-in under bwrap; every new fence has a test.
2026-11-19 14:43:26 -06:00
jason.woltje fbaae25d06 fix(webui): settings slice 1: no harness names written into the page: Settings > Triage shows each harness by the id its module is installed under, from the route read, and its help line says the list holds only the programs the model runs with; the drawer's harness refusal reads the route and names its harness by that id, or keeps a plain line without a name when the route can't be read (#1873, lane settings-s1) 2026-11-19 14:38:04 -06:00
jason.woltje c9d64a341e fix(webui): settings slice 1: the failed settings read and failedView share one copy of the line every failed read ends with, so the shipped-strings pin in shell.test finds it once again (#1873, lane settings-s1) 2026-11-19 14:04:14 -06:00
jason.woltje 4cbf3f07b0 fix(webui): settings slice 1 review round 2: a model-key-collision catalogue says two models share a short name and offers no Restart, so it no longer holds Model setup's next step; the dev fixture line says it can't be changed from Console yet instead of asking for a Routing change that does not remove the fixture; Model setup owner text says model service in the init lock, repair and fence steps, and the fence clear command sits under Technical details with where to run it; the Health list and the stopped-organization page show the reason the workspace read already carries; the stopped page links to Model setup by its name; the hire start refusals point at Settings > Routing for models (#1873, lane settings-s1) 2026-11-19 14:04:14 -06:00
jason.woltje 48a2ed675c fix(webui): settings slice 1 review round: Model setup reads each organization's own routes and offers Restart only where the bus names no cause or one a restart clears, so the dev fixture gets its plain line and a Routing link and only a restartable organization holds the next step; the bus's cause codes read as plain lines with its words under Technical details, and the lock cause gets a primary Restart; a set fence reads as a restart to finish; a failed status read offers Check again; Save routes steps back while Restart is the primary; the page is named Model setup and owner text says model service; Routing explains harness and preflight; the drawer's harness refusal asks for a model that runs with Claude Code; the Triage page says daily token limit and that Triage also needs its key (#1873, lane settings-s1) 2026-11-19 14:04:14 -06:00
jason.woltje e70b6bd684 feat(webui): settings slice 1: Routing says why an organization can't see your models: the model service is not set up or not running (a link to its page), the bus names its cause (that line and Check again, plus Restart only for the lock cause), or the service runs and the bus names none (Restart); Model proxy keeps a Restart per organization whose catalogue is not ready, rereads the whole status after a sign-in, works out the steps after accounts from accounts, organizations and the Triage route, and shows only the next step's button with the rest under Technical details; Triage refusals are plain lines with a link to the page that fixes them, and the harness refusal names Claude Code; a stopped organization keeps its refused restart on screen and keeps the instance pages reachable; at phone width a Routing model select takes the whole row (#1873, lane settings-s1) 2026-11-19 14:03:27 -06:00
263 changed files with 27449 additions and 1661 deletions

No files matched your search

+4 -15
View File
@@ -1,24 +1,13 @@
# Non-secret runtime settings for the mosaic-poc-agent container.
# Copy to .env if you want to override the defaults in compose.yaml.
#
# NEVER put credentials in this file. Authentication is supplied at
# runtime only, via one of the two documented paths:
# 1. read-only mounted pi auth file (default: ~/.pi/agent/auth.json,
# override the host path with PI_AUTH_FILE)
# 2. provider API key environment variable (ZAI_API_KEY or
# ANTHROPIC_API_KEY), passed through by compose.yaml when set
# NEVER put credentials in this file. A worker holds no credential of
# its own: it reaches its model through the local model proxy with the
# worker key Mosaic issues (Settings > Model setup). No pi auth file is
# mounted and no provider key variable is passed through.
# Model provider (built-in pi provider name)
PI_PROVIDER=zai
# Model ID within the provider
PI_MODEL=glm-5.3-flash
# Optional: alternative host path of the pi credential file mounted
# read-only at /home/node/.pi/agent/auth.json in the container
#PI_AUTH_FILE=/home/jwoltje/.pi/agent/auth.json
# Optional: documented env-var auth alternative (secret! set in your
# shell or a gitignored .env, never commit)
#ZAI_API_KEY=
#ANTHROPIC_API_KEY=
+14 -4
View File
@@ -222,8 +222,18 @@ Compacted, restarted, or new? Nothing that matters is lost: this file +
`git log --oneline -10` + the suites reconstruct the full state. **Never
guess** — verify with the suites; the run records and logs hold the receipts.
## Version pin
## Harness versions
`@earendil-works/pi-coding-agent` is pinned exactly (see `package.json` /
`RELEASE`); never install unversioned. Release identity: `RELEASE` file
(0.0.X until declared stable); image tags derive from it.
Harness binaries are not pinned (Jason, 2026-11-19). pi runs from
`MOSAIC_PI_BIN`, else the first `pi` on PATH, else the repository's
`node_modules`. `package.json` holds a floor range, and `package-lock.json`
records the version `npm ci` installs and the worker image carries. Nothing
refuses a pi for its version. Before a session or conversation starts, a
fit check proves the flags and behaviours Mosaic needs and refuses with a
plain line when one is missing. The bus host repeats the check on a timer
and when a binary changes. The version and a digest of the files that
ran are recorded in the claim and the run record. A change under a live
claim still refuses. Never install unversioned: a lock change is a
reviewed commit that passes the fit check and the suites. Release
identity: `RELEASE` file (0.0.X until declared stable). Image tags derive
from it and the lock's resolved pi version.
+8 -4
View File
@@ -15,8 +15,11 @@ RUN apt-get update \
# uid/gid 1000, which matches the host user that owns the runtime
# state directory mounted at /var/lib/mosaic. It is reused as-is.
# Pinned Pi install: package.json pins the exact version and
# package-lock.json is installed with npm ci. No unversioned installs.
# Pi install: package.json gives pi a range, not a pin (#1873), and npm ci
# installs the version package-lock.json resolves. No unversioned installs.
# The build entry points run scripts/pi-fit-image.mjs on the host first, so a
# lock that resolves an unfit pi fails the build (the fit check needs bwrap,
# which docker build's seccomp profile refuses, so it is not a RUN step).
WORKDIR /opt/app
COPY package.json package-lock.json ./
RUN npm ci --ignore-scripts
@@ -32,8 +35,9 @@ RUN chmod 0555 /opt/mosaic/contracts /opt/mosaic/contracts/* \
&& chmod 0555 /opt/mosaic/adapters /opt/mosaic/adapters/*/adapter.sh
# Private per-container prompt/output paths, the fixed narrow-mount targets,
# workspace, and pi agent directory (auth.json is bind-mounted read-only at
# runtime; no host data root is mounted).
# workspace, and pi agent directory (worker-cutover: no credential file is mounted;
# a pi worker's models.json, which names the model proxy and holds no key, is
# bind-mounted read-only at runtime; no host data root is mounted).
RUN mkdir -p /var/lib/mosaic/private /var/lib/mosaic/input/user /var/lib/mosaic/input/skills /var/lib/mosaic/control /var/lib/mosaic/session /workspace /home/node/.pi/agent \
&& touch /var/lib/mosaic/input/task.json /var/lib/mosaic/input/mission.json /var/lib/mosaic/input/agent-soul.md /var/lib/mosaic/input/session-fork.jsonl /var/lib/mosaic/control/processing.json \
&& chown -R node:node /var/lib/mosaic /workspace /home/node /opt/app \
+6 -4
View File
@@ -6,13 +6,15 @@ not implemented (see BRIEF.md, "Explicit exclusions").
## L0 — Implemented: container returns MOSAIC_HELLO_OK
One image (`mosaic-poc-agent:0.84.4`, built on digest-pinned `node:24.22.2-bookworm-slim`, non-root,
pinned Pi) runs one Pi agent one-shot. Four immutable local contract files are
One image (`mosaic-poc-agent:<pi>-r<release>`, the pi version `package-lock.json` resolves; built on digest-pinned `node:24.22.2-bookworm-slim`, non-root,
pi from the lock, not pinned, and fit-checked before the build) runs one Pi agent one-shot. Four immutable local contract files are
loaded in fixed order into the generated system prompt
(`/var/lib/mosaic/private/system-prompt.md`, private to that container). One real model request is sent
noninteractively; the response must equal `MOSAIC_HELLO_OK` exactly or the
verification exits nonzero. Authentication is supplied at runtime only
(read-only mounted pi auth file, or a provider API key environment variable).
verification exits nonzero. Authentication is supplied at runtime only: since
worker-cutover a worker reaches its model through the local model proxy, with
the worker key Mosaic issues passed only as `ANTHROPIC_API_KEY` and the relay
socket directory mounted read-only. No pi auth file is mounted.
## L1 — Deferred: persist and resume a named Pi session
+23 -13
View File
@@ -193,7 +193,14 @@ explicit managed-name refusals with migration compatibility.
## How it works
1. `scripts/build.sh` builds the release image (`mosaic-poc-agent:<pi>-r<release>`,
tag derived from `RELEASE` + the pinned Pi version) with Docker Compose.
tag derived from `RELEASE` + the pi version `package-lock.json` resolves, by
`scripts/image-tag.mjs`) with `docker build -f Containerfile`. `package.json`
gives pi a range, never a pin. Before the build, `scripts/pi-fit-image.mjs`
runs the fit check on the pi the lock resolves, so a lock that resolves an
unfit pi fails it. The check runs on the host's node, which can differ from
the image's (the Containerfile's `FROM`). `compose.yaml` has no build section
and `pull_policy: never`, so a compose run never builds or pulls the image;
it refuses a missing one.
2. On each run, `/opt/mosaic/src/load-contracts.sh` reads the four contract files
in fixed order (CONSTITUTION, STANDARDS, SOUL, USER), joins them with clear
separators, and writes `/var/lib/mosaic/private/system-prompt.md` in the
@@ -230,21 +237,24 @@ EXPECTED_MARKER=MOSAIC_NOT_OK scripts/verify.sh # must exit nonzero
## Authentication
Pi's documented container authentication (see the package's
`docs/containerization.md`) is used, in this order:
A worker container holds no credential of its own (worker-cutover). It
reaches its model only through the instance's local model proxy, with the
worker key Mosaic issues, re-adds and rotates itself at launch. Each launch
passes the key to the container only as `ANTHROPIC_API_KEY`, and mounts the
relay socket directory read-only at `/run/mosaic-proxy` and a `models.json`
that names the proxy (and holds no key) read-only at
`/home/node/.pi/agent/models.json`. No pi auth file is mounted, and a
caller's `ZAI_API_KEY` or `ANTHROPIC_API_KEY` never reaches a worker.
1. **Read-only mounted credential file** (default): the host pi auth file
`~/.pi/agent/auth.json` is bind-mounted read-only to
`/home/node/.pi/agent/auth.json`. The host file holds a static API-key
entry for the built-in `zai` provider, so no token refresh writes are needed.
2. **Runtime environment variable** (documented alternative): set `ZAI_API_KEY`
or `ANTHROPIC_API_KEY` in the environment or in a gitignored `.env`; compose
passes them through. Pi's documented precedence applies.
Set up once: start the proxy and add an account on Settings > Model setup.
The worker's provider and model stay `execution.provider` and
`execution.model` in the config; a Claude model is refused for a pi worker.
The launch checks and their one-line refusals are in `docs/TOOLS.md`.
Credentials are never committed, never copied into the image, and never printed.
Mosaic-managed named accounts (`agent.sh --auth`) live under the data root
(`auth/<account>.json`, 0600) — the stack never writes into `~/.pi`.
`.env.example` contains non-secret settings only.
The key's text is redacted from run evidence; a run record keeps only
`modelKeyId`. The stack never writes into `~/.pi`. `.env.example` contains
non-secret settings only.
## Boundaries honored
+1 -1
View File
@@ -1 +1 @@
0.0.13
0.0.14
+26 -2
View File
@@ -31,6 +31,9 @@ Optional, adapter-specific (documented per adapter):
| Variable | Meaning |
|---|---|
| `MOSAIC_MOCK_RESPONSE` | mock only: the verbatim response to emit |
| `MOSAIC_PI_FINAL_STOP` | pi only, `1` from `src/run-agent.sh` for a headless worker: judge the run by pi's final stop reason (`final-stop.mjs`) |
| `MOSAIC_PROXY_PORT` | pi only, set by the host launcher (worker-cutover): start `proxy-forward.mjs` on `127.0.0.1:<port>` against the relay socket mounted at `/run/mosaic-proxy/proxy.sock`. The adapter refuses (exit 1) unless the forwarder's captured stdout is exactly `ready`. Unset: no proxy and no credential, so pi fails on its own. |
| `ANTHROPIC_API_KEY` | pi only: the worker key Mosaic issued for this launch, read by the mounted `models.json` as `$ANTHROPIC_API_KEY`. It is the container's only credential; the adapter never prints it. Empty for the mock adapter. |
**Outputs:**
@@ -52,8 +55,11 @@ Optional, adapter-specific (documented per adapter):
## Included adapters
- `pi` — the pinned `@earendil-works/pi-coding-agent` CLI in noninteractive
print mode (`-p`), ambient discovery disabled, stdin detached.
- `pi` — the `@earendil-works/pi-coding-agent` CLI (not pinned; #1873) in noninteractive
print mode (`-p`), ambient discovery disabled, stdin detached. A worker run
(`MOSAIC_PI_FINAL_STOP=1`) goes through `final-stop.mjs`: pi runs with
`--mode json`, the answer is printed only on a finished stop, and an error,
aborted or output-limit stop exits 1 with one line on stderr.
- `mock` — deterministic echo of `MOSAIC_MOCK_RESPONSE`. Test-only: never use
it where a real model response is required.
- `claude` — the host's `claude` CLI, for managed sessions only (below):
@@ -106,6 +112,24 @@ keeps its existing channel; the CLI does not inherit the completion pipe. No ext
writable mount or completion file is needed. Pi retains print mode and signals
activity through its existing extension. Mock remains a test-only echo.
### A worker's skills (M17, #1873)
`MOSAIC_SKILLS` is a comma list of skill directories. Today only the
interactive container launch (`scripts/agent.sh`) sets it; task launches
refuse skill directories and set it empty
(`packages/processing-control/src/containers.mjs`). The pi adapter passes `--no-skills` and then
`--skill <dir>` for each one. On pi 0.85.1 `--skill` adds to discovery and
does not replace it. With `--skill` alone, a probe with a scratch HOME also
loaded the user's skills, the skills a `settings.json` or a listed package
names, and `~/.agents/skills` (and the project's, without `--no-approve`).
With `--no-skills` first, only the given directories load. A directory that
doesn't exist stops the adapter with exit 2. The fit check runs the
headless form of this argv, with two skill directories, one `-e` file, no
tools and no session, as its last `-p` run (`workerShape`,
`packages/harness/README.md`); the TUI shape the launch uses is not run. The fit check runs
before bus sessions, and on the pi `package-lock.json` resolves before an
image build (`scripts/pi-fit-image.mjs`); worker containers run that pi.
### A role's skills (row 64)
A managed session's skills reach both adapters as text in the bundle's
+41 -8
View File
@@ -1,6 +1,6 @@
#!/bin/sh
# Pi adapter: implements the Mosaic adapter contract for the pinned
# @earendil-works/pi-coding-agent CLI.
# Pi adapter: implements the Mosaic adapter contract for the
# @earendil-works/pi-coding-agent CLI. Its behaviour notes are read from 0.85.1.
#
# Contract: see /opt/mosaic/adapters/README.md.
# Headless (default): stdout = response only; stderr = diagnostics; exit 0.
@@ -25,6 +25,18 @@ esac
: "${PI_PROVIDER:?pi adapter: PI_PROVIDER is required}"
: "${PI_MODEL:?pi adapter: PI_MODEL is required}"
# Model proxy (worker-cutover, S4): the host launcher (packages/processing-control/src/
# worker-proxy.mjs) mounts its relay's socket dir at /run/mosaic-proxy and sets
# MOSAIC_PROXY_PORT. The forwarder listens on 127.0.0.1:$MOSAIC_PROXY_PORT, where the mounted
# models.json points pi, after one request through the socket got a reply. It prints exactly
# "ready" on stdout and closes it, so the capture below returns while it keeps running. Anything
# else refuses: the forwarder already printed its one line on stderr. With no port set there is
# no proxy and no credential in this container, so pi fails on its own.
if [ -n "${MOSAIC_PROXY_PORT:-}" ]; then
ready=$(node "$(dirname "$0")/proxy-forward.mjs" /run/mosaic-proxy/proxy.sock "$MOSAIC_PROXY_PORT" &)
[ "$ready" = "ready" ] || exit 1
fi
INTERACTIVE="${MOSAIC_INTERACTIVE:-}"
if [ "$INTERACTIVE" != "1" ]; then
[ -n "${MOSAIC_REQUEST:-}" ] || { echo "pi adapter: MOSAIC_REQUEST is required" >&2; exit 2; }
@@ -57,14 +69,18 @@ fi
TOOLS_FLAG="--no-tools"
[ -n "${MOSAIC_TOOLS:-}" ] && TOOLS_FLAG="--tools $MOSAIC_TOOLS"
# Skills (M17): explicitly provided skill dirs replace discovery. When none
# are provided the agent runs with --no-skills (nothing ambient to find).
# Skills (M17, #1873 pi-skills): --no-skills always, then one --skill per
# given dir. On pi 0.85.1 --skill adds to discovery and does not replace it:
# with --skill alone pi also loads the user's skills, settings and package
# skills and ~/.agents/skills (a planted-skill probe showed it, offline, with a
# scratch HOME). With --no-skills first only the given dirs load. The fit check
# (packages/harness/src/pi-fit.mjs) runs its -p form as its last run (workerShape).
# Only the interactive launch (scripts/agent.sh) sets MOSAIC_SKILLS today.
# Role sessions never get MOSAIC_SKILLS (ENV_ALLOW in packages/seat/src/
# session.mjs leaves it out), so they run with --no-skills; their skills
# session.mjs leaves it out), so they run with --no-skills alone; their skills
# are text in the system prompt (row 64).
SKILLS_FLAG="--no-skills"
if [ -n "${MOSAIC_SKILLS:-}" ]; then
SKILLS_FLAG=""
OLDIFS=$IFS; IFS=','
for s in $MOSAIC_SKILLS; do
[ -d "$s" ] || { echo "pi adapter: skill dir missing: $s" >&2; exit 2; }
@@ -92,6 +108,11 @@ fi
THINKING_FLAG=""
[ -n "${MOSAIC_THINKING:-}" ] && THINKING_FLAG="--thinking $MOSAIC_THINKING"
# MCP (#1873 S3): the harness module sets MOSAIC_PI_NO_MCP=1 when the binary's --help lists
# --no-mcp, so MCP stays off by its own flag and not only as a side effect of --no-extensions.
NO_MCP_FLAG=""
[ "${MOSAIC_PI_NO_MCP:-}" = "1" ] && NO_MCP_FLAG="--no-mcp"
# Mode (M13): interactive TUI or one-shot print.
PRINT_MODE="-p"
REQUEST_ARG=""
@@ -107,6 +128,8 @@ fi
# --system-prompt replace the default prompt with the generated one, given
# as the file's absolute path (checked above), never its text
# --no-* no ambient context/skills/extensions/templates/themes
# SKILLS_FLAG --no-skills, then --skill <dir> for each MOSAIC_SKILLS dir
# NO_MCP_FLAG --no-mcp when the binary lists it, else nothing
# EXT_FLAGS the explicitly provided extensions only (-e)
# SESSION_FLAGS ephemeral | persistent | forked (per env)
# TOOLS_FLAG per capabilities
@@ -118,6 +141,7 @@ set -- \
--offline \
--no-approve \
--no-extensions \
$NO_MCP_FLAG \
$EXT_FLAGS \
$SKILLS_FLAG \
--no-prompt-templates \
@@ -134,6 +158,15 @@ set -- \
[ "$INTERACTIVE" = "1" ] || set -- "$@" -p "$MOSAIC_REQUEST"
if [ -n "${MOSAIC_MODEL_EVENT_MONITOR:-}" ] && [ "$INTERACTIVE" != "1" ]; then
STREAM="$(dirname "$0")/../../packages/plugins/harness/pi/model-stream.mjs"
exec node "$STREAM" pi "$MOSAIC_FIRST_OUTPUT_DIR" pi "$@" --mode json
exec node "$STREAM" pi "$MOSAIC_FIRST_OUTPUT_DIR" "${MOSAIC_PI_BIN:-pi}" "$@" --mode json
fi
exec pi "$@"
# worker-stop-reason: a run-task worker (src/run-agent.sh sets MOSAIC_PI_FINAL_STOP=1) is judged
# by pi's final stop reason as well as its exit status. Text mode exits 0 on a "length" stop with
# nothing printed; final-stop.mjs runs pi with --mode json, prints the answer as text mode does on
# a finished stop, and fails the run on an error, aborted or length stop with one line on stderr.
# Managed sessions never set it and keep text mode.
if [ "${MOSAIC_PI_FINAL_STOP:-}" = "1" ] && [ "$INTERACTIVE" != "1" ]; then
exec node "$(dirname "$0")/final-stop.mjs" "${MOSAIC_PI_BIN:-pi}" "$@"
fi
# #1873 S2: the binary the harness module found (MOSAIC_PI_BIN, its path inside the sandbox).
exec "${MOSAIC_PI_BIN:-pi}" "$@"
+152
View File
@@ -0,0 +1,152 @@
#!/usr/bin/env node
// worker-stop-reason: judge a headless pi worker run by pi's final stop reason, not only its exit
// status. The worker entrypoint (src/run-agent.sh) asks for this with MOSAIC_PI_FINAL_STOP=1, and
// adapter.sh then runs `node final-stop.mjs <pi> <args...>` in place of `<pi> <args...>`.
//
// Why: in text print mode (-p) pi 0.85.1 exits 1 on an "error" or "aborted" stop, but a "length"
// stop (the model reached its output limit) exits 0 with nothing on stdout or stderr, so run-task
// recorded an empty success. Only the JSON event stream says how the run ended: its last
// agent_end event carries the run's messages, and the last assistant message's stopReason is the
// final stop. Proved against pi 0.85.1 with a fake provider on loopback (packages/plugins/tests/
// pi-final-stop.test.mjs).
//
// This runs pi with `--mode json` and keeps only the last agent_end line (bounded). Then:
// pi exited nonzero the same code, pi's own stderr already said why
// pi was stopped by a signal 128 + the signal number, one line on stderr
// final stop "stop" the text parts, each followed by "\n" (as text mode prints), exit 0
// final stop "toolUse" nothing, exit 0 (a tool ended the run; text mode prints nothing)
// final stop "error"/"aborted" exit 1, one line: pi stopped on an error: <pi's error message>
// final stop "length" exit 1, one line: the model reached its output limit
// anything else exit 1, one line (fail closed: no final answer to judge)
// stdout carries the answer only on success. An error line carries pi's error message as the
// provider sent it, as text mode printed it; run-task redacts known secret values before it shows
// the cause (runCause in scripts/mosaic-task.mjs).
import { spawn } from "node:child_process";
import { realpathSync } from "node:fs";
import { constants } from "node:os";
import { fileURLToPath } from "node:url";
export const AGENT_END_PREFIX = '{"type":"agent_end"';
// The cap on the one line kept. agent_end carries every message of the run, tool results too.
export const MAX_AGENT_END_BYTES = 64 * 1024 * 1024;
const MAX_LINE_CHARS = 2000;
const oneLine = (text) => {
const line = String(text).replace(/\s+/g, " ").trim();
return line.length > MAX_LINE_CHARS ? `${line.slice(0, MAX_LINE_CHARS)}...` : line;
};
// The judgement for the last agent_end line pi wrote (null when it wrote none):
// { ok: true, answer } or { ok: false, message }.
export function judgeFinalStop(agentEndLine) {
if (agentEndLine === null) return { ok: false, message: "pi ended without a final answer: it reported no end of run" };
let event;
try { event = JSON.parse(agentEndLine); } catch { return { ok: false, message: "pi ended without a final answer: its end of run could not be read" }; }
const messages = Array.isArray(event?.messages) ? event.messages : [];
let last = null;
for (let i = messages.length - 1; i >= 0; i--) {
if (messages[i]?.role === "assistant") { last = messages[i]; break; }
}
if (!last) return { ok: false, message: "pi ended without a final answer: the model gave no reply" };
const reason = last.stopReason;
if (reason === "stop" || reason === "toolUse") {
if (messages[messages.length - 1] !== last) return { ok: true, answer: "" };
const parts = Array.isArray(last.content) ? last.content : [];
return { ok: true, answer: parts.filter((p) => p?.type === "text" && typeof p.text === "string").map((p) => `${p.text}\n`).join("") };
}
if (reason === "error" || reason === "aborted") {
const said = typeof last.errorMessage === "string" && last.errorMessage.trim() ? oneLine(last.errorMessage) : `request ${reason}`;
return { ok: false, message: `pi stopped on an error: ${said}` };
}
// A worker run fails on "length" even with partial text: the answer was cut off. A managed
// session first turn (packages/plugins/harness/pi/model-stream.mjs) counts length with text as
// output; that turn is a conversation, a worker run is a finished job.
if (reason === "length") return { ok: false, message: "pi stopped with no finished answer: the model reached its output limit" };
return { ok: false, message: `pi ended without a final answer: it stopped for an unknown reason (${oneLine(String(reason)).slice(0, 40)})` };
}
// Keeps the last line that starts with AGENT_END_PREFIX out of a byte stream, holding at most
// one such line (up to max bytes) in memory. Other lines pass through unkept.
export function agentEndKeeper(max = MAX_AGENT_END_BYTES) {
let chunks = [];
let size = 0;
let state = "start"; // start: at a line start; keep: in an agent_end line; skip: in another line
let head = Buffer.alloc(0);
let last = null;
let tooBig = false;
const finishLine = () => {
if (state === "keep") {
if (size > max) tooBig = true;
else { last = Buffer.concat(chunks).toString("utf8"); tooBig = false; }
}
chunks = []; size = 0; state = "start"; head = Buffer.alloc(0);
};
const feedPart = (part) => {
if (state === "start") {
head = Buffer.concat([head, part]);
const prefix = Buffer.from(AGENT_END_PREFIX);
const n = Math.min(head.length, prefix.length);
if (!head.subarray(0, n).equals(prefix.subarray(0, n))) { state = "skip"; head = Buffer.alloc(0); return; }
if (head.length < prefix.length) return;
state = "keep"; part = head; head = Buffer.alloc(0);
}
if (state === "keep") {
size += part.length;
if (size <= max) chunks.push(part); else chunks = [];
}
};
return {
write(buffer) {
let at = 0;
while (at < buffer.length) {
const nl = buffer.indexOf(10, at);
if (nl === -1) { feedPart(buffer.subarray(at)); return; }
feedPart(buffer.subarray(at, nl));
finishLine();
at = nl + 1;
}
},
end() {
if (state === "keep") finishLine();
return { line: tooBig ? null : last, tooBig };
},
};
}
async function main(argv) {
const [pi, ...args] = argv;
if (!pi) { process.stderr.write("pi final stop: no pi binary was given\n"); return 2; }
const keeper = agentEndKeeper();
const child = spawn(pi, [...args, "--mode", "json"], { stdio: ["inherit", "pipe", "inherit"] });
const forward = (signal) => { try { child.kill(signal); } catch {} };
for (const signal of ["SIGTERM", "SIGINT", "SIGHUP"]) process.on(signal, () => forward(signal));
child.stdout.on("data", (buffer) => keeper.write(buffer));
const outcome = await new Promise((resolve) => {
child.on("error", (error) => resolve({ error }));
child.on("close", (code, signal) => resolve({ code, signal }));
});
if (outcome.error) {
process.stderr.write(`pi final stop: pi could not be started (${outcome.error.code ?? "spawn error"})\n`);
return 127;
}
if (outcome.signal) {
process.stderr.write(`pi final stop: pi was stopped by ${outcome.signal}\n`);
const number = constants.signals[outcome.signal];
return Number.isInteger(number) && number > 0 ? 128 + number : 1;
}
if (outcome.code !== 0) return outcome.code ?? 1;
const { line, tooBig } = keeper.end();
const verdict = tooBig
? { ok: false, message: `pi ended without a final answer: its end of run was over ${MAX_AGENT_END_BYTES / (1024 * 1024)} MiB` }
: judgeFinalStop(line);
if (!verdict.ok) {
process.stderr.write(`${verdict.message}\n`);
return 1;
}
await new Promise((resolve) => process.stdout.write(verdict.answer, resolve));
return 0;
}
let isMain = false;
try { isMain = realpathSync(process.argv[1] ?? "") === realpathSync(fileURLToPath(import.meta.url)); } catch {}
if (isMain) process.exitCode = await main(process.argv.slice(2));
+124
View File
@@ -0,0 +1,124 @@
#!/usr/bin/env node
// In-container forwarder for the model proxy (worker-proxy-simple, slice S2).
//
// node proxy-forward.mjs <socket path> <port>
//
// Listens on 127.0.0.1:<port> inside the container and copies bytes, both
// ways, to the host relay's Unix socket (packages/processing-control/src/
// proxy-relay.mjs), which copies them to the host's loopback model proxy.
// It parses nothing and holds no key.
//
// Start check. Before it listens it makes one connection through the socket
// and sends one small HTTP request (HEAD /). Any reply byte within five
// seconds passes: the socket, the relay and the proxy all answered. Anything
// else prints exactly one line on stderr and exits 1:
//
// worker: the container could not reach the model proxy socket.
//
// The same line is printed once, and the forwarder keeps running, when a later
// connection fails before any reply byte: the socket refuses, or the relay
// hangs up on a request with no reply, which is what a gone proxy looks like.
// An idle connection that the proxy closes before any request is not counted.
//
// Output. One line "ready" on stdout once the port listens, then stdout is
// closed. The caller captures it (for example ready=$(node ... &)), so it
// never mixes with the adapter's response on stdout. A usage or listen
// failure is one other line on stderr and exits 2.
// Not called by adapter.sh yet; the cutover slice (S4) starts it.
import { closeSync, openSync } from 'node:fs';
import net from 'node:net';
import { isAbsolute } from 'node:path';
const UNREACHABLE = 'worker: the container could not reach the model proxy socket.';
const CHECK_MS = 5000;
const CHECK_REQUEST = 'HEAD / HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n';
let told = false;
function unreachable() {
if (!told) { told = true; process.stderr.write(`${UNREACHABLE}\n`); }
}
function check(socketPath) {
return new Promise((resolveCheck) => {
let passed = false, finished = false;
const probe = net.connect({ path: socketPath });
const finish = (ok) => {
if (finished) return;
finished = true;
clearTimeout(timer);
probe.destroy();
resolveCheck(ok);
};
const timer = setTimeout(() => finish(false), CHECK_MS);
probe.once('connect', () => probe.write(CHECK_REQUEST));
probe.on('data', () => { passed = true; finish(true); });
probe.on('error', () => finish(false));
probe.on('close', () => finish(passed));
});
}
// Copy bytes both ways. An error on either side ends both at once. A clean
// close of one side only ends the other, so bytes still queued for a slow
// reader are flushed first; that side is destroyed when its peer closes too,
// or LINGER_MS after its last byte was written. Kept in step with the same
// helper in packages/processing-control/src/proxy-relay.mjs.
const LINGER_MS = 10000;
function join(a, b) {
const kill = () => { a.destroy(); b.destroy(); };
const finish = (other) => () => {
if (other.destroyed) return;
other.resume();
other.end();
const linger = () => {
const timer = setTimeout(() => other.destroy(), LINGER_MS).unref();
other.once('close', () => clearTimeout(timer));
};
if (other.writableFinished) linger(); else other.once('finish', linger);
};
a.on('error', kill); b.on('error', kill);
a.on('close', finish(b)); b.on('close', finish(a));
a.pipe(b); b.pipe(a);
}
async function main(argv) {
const port = /^[1-9][0-9]{0,4}$/.test(argv[1] ?? '') ? Number(argv[1]) : 0;
if (argv.length !== 2 || !isAbsolute(argv[0]) || port < 1 || port > 65535) {
process.stderr.write('worker: proxy forwarder usage: proxy-forward.mjs <socket path> <port>.\n');
process.exit(2);
}
const socketPath = argv[0];
if (!(await check(socketPath))) {
unreachable();
process.exit(1);
}
const server = net.createServer({ allowHalfOpen: true }, (client) => {
const upstream = net.connect({ path: socketPath, allowHalfOpen: true });
let clientDone = false;
client.once('end', () => { clientDone = true; });
client.once('close', () => { clientDone = true; });
// The socket failed, or the relay hung up on a request before any reply
// byte while the client still waited: what a gone proxy looks like. An
// idle connection the proxy closes before any request is not counted.
const lost = () => { if (!clientDone && upstream.bytesRead === 0) unreachable(); };
upstream.on('error', lost);
upstream.once('end', () => { if (upstream.bytesWritten > 0) lost(); });
join(client, upstream);
});
server.on('error', () => {
process.stderr.write(`worker: the model proxy forwarder could not listen on 127.0.0.1:${port}.\n`);
process.exit(2);
});
server.listen({ host: '127.0.0.1', port, exclusive: true }, () => {
// Node never closes fd 1 itself, so close it and put /dev/null in its place:
// a caller reading stdout to its end then gets EOF right after "ready".
process.stdout.write('ready\n', () => {
try { closeSync(1); openSync('/dev/null', 'w'); } catch {}
});
});
}
main(process.argv.slice(2)).catch(() => {
unreachable();
process.exit(1);
});
+12 -11
View File
@@ -1,8 +1,10 @@
services:
mosaic-agent:
build:
context: .
dockerfile: Containerfile
# #1873 S5: no build section, and never pulled. The image is built only by
# build_image (scripts/common.sh) or the gate's prepareImage, which run the
# image fit step first, so a missing image refuses here instead of compose
# building one that skipped the fit check.
pull_policy: never
image: ${MOSAIC_IMAGE_TAG:?MOSAIC_IMAGE_TAG must be set by scripts/load_release (run via scripts/*.sh)}
user: "1000:1000"
environment:
@@ -24,15 +26,14 @@ services:
MOSAIC_AGENT_ROLE: ${MOSAIC_AGENT_ROLE:-}
# mock adapter only: verbatim response for deterministic seam tests
MOSAIC_MOCK_RESPONSE: ${MOSAIC_MOCK_RESPONSE:-}
# Documented container auth alternative: provider API key via
# runtime environment variable. Empty by default; when empty Pi
# falls back to the read-only mounted auth.json credential file.
ZAI_API_KEY: ${ZAI_API_KEY:-}
# worker-cutover: a pi worker reaches its model only through the
# instance's model proxy. The launcher (packages/processing-control/
# src/worker-proxy.mjs) sets both from the worker key it issued and
# the relay it started; it clears them for every other launch, so a
# caller's own provider key never enters the container. There is no
# credential file mount.
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
volumes:
# Runtime credential only: pi auth file mounted READ-ONLY.
# Never copied into the image.
- ${PI_AUTH_FILE:-/home/jwoltje/.pi/agent/auth.json}:/home/node/.pi/agent/auth.json:ro
MOSAIC_PROXY_PORT: ${MOSAIC_PROXY_PORT:-}
# Headless runs: the request is passed as command args by the launchers
# (run-task.sh) or defaults inside run-agent.sh (hello/verify). Never a
# fixed command here - interactive runs (scripts/agent.sh) need no args.
+152 -16
View File
@@ -16,7 +16,7 @@ and changes nothing.
| Command | Purpose | Notes |
|---|---|---|
| `scripts/bootstrap.sh [--instance NAME]` | Create `~/.config/mosaic-dev/config.json` if absent; with `--instance NAME`, `~/.config/mosaic-NAME/config.json` and the claimed root `~/.mosaic-NAME` | Idempotent; existing config validated, never rewritten. See [Stack instances](#stack-instances---instance-name-row-190) |
| `scripts/build.sh` | Build the release image | Tag derived from `RELEASE` + pinned pi version |
| `scripts/build.sh` | Build the release image | Tag `mosaic-poc-agent:<pi>-r<RELEASE>`, pi being the version `package-lock.json` resolves (`scripts/image-tag.mjs`); runs the image fit step (`scripts/pi-fit-image.mjs`) first. compose runs never build or pull the image, so run this before anything that needs a new tag |
| `scripts/hello.sh` | One-shot startup request | Prints model response on stdout |
| `scripts/verify.sh` | Full gated test | Exit 0 only on exact `MOSAIC_HELLO_OK`; `EXPECTED_MARKER` overrides for negative drills |
@@ -39,6 +39,86 @@ Task fields: `prompt` (required), `mission` (path), `expectExact`,
`directives[]`, optional governing `capabilities.tools`. Policy: a task may
narrow a mission's tools, never widen; empty intersection = tool-free run.
Worker launch checks (worker-cutover). A run's provider and model come only
from the instance config, `execution.provider` and `execution.model`; task and
mission files have no provider field. A worker container holds no credential
of its own. Every host worker launch (`mosaic-task run` and `retry`, so
`run-task.sh` and the release health check, and `scripts/container.sh` for
`verify.sh`, `hello.sh` and `agent.sh`) goes through
`packages/processing-control/src/worker-proxy.mjs`. For a pi worker it checks,
before a run directory exists and before docker:
1. the host user is uid 1000 (the image's user);
2. the model proxy is set up (`proxy.json` beside the config);
3. the model is not a Claude model (Claude OAuth models run only in the
Claude harness);
4. the management key loads and the proxy's model list has an account that
serves `execution.provider/execution.model` for pi.
It then issues, re-adds or rotates the worker key (`msw-<tag>-worker-…`, in
`<configDir>/secrets/worker/`) under the proxy lock when needed, and reads it,
reading again when another launch rotated it in between. Each refusal is one
line on stderr, exit 3, and no run record:
| Failure | Line |
|---|---|
| Proxy not set up | `proxy-not-configured: The model proxy is not set up. Set it up on Settings > Model setup.` |
| Proxy not running | `proxy-down: The model proxy is not running. Start it on Settings > Model setup.` |
| No account for the model | `model-provider-unavailable: No account serves zai/glm-5.3-flash. Models you can use: (list). Add an account on Settings > Model setup, or set execution.model in the config.` |
| Model switched off | `model-provider-unavailable: zai/glm-5.3-flash is switched off. Switch it on in Settings > Model setup.` |
| Claude model on pi | `model-harness-incompatible: Claude models run only in the Claude harness. Pick another model in the config.` |
| Key refused | `proxy-key-refused: The proxy did not accept the worker key. Restart the proxy on Settings > Model setup, then run verify again.` |
| Another key write running | `proxy-lock-busy: Another proxy key change is running. Run verify again in a minute.` |
| Proxy write uncertain | `proxy-uncertain: A proxy key change did not finish. Restart the proxy, then run \`mosaic proxy fence clear --proxy-restarted\`.` |
| Host user is not uid 1000 | `Worker containers run as user 1000. Run Mosaic as that user.` (no code) |
`mosaic-task` prefixes a coded line with `mosaic-task: `, `container.sh` with
`processing: `. Rarer `worker-key-*` codes come from the key module with its
own fixed sentence.
The launch then registers the runtime record, makes
`<dataRoot>/worker-proxy/<id>/` (0700, `<id>` is 12 hex), writes pi's
`models.json` there (it names the proxy and reads the key from
`$ANTHROPIC_API_KEY`; it holds no key), and starts the relay at
`<id>/proxy.sock`. Docker gets the key only in `ANTHROPIC_API_KEY`, the
directory read-only at `/run/mosaic-proxy` and `models.json` read-only at
`/home/node/.pi/agent/models.json`. No pi auth file is mounted. The caller's
`ZAI_API_KEY` is removed and its `ANTHROPIC_API_KEY` replaced for every
adapter. When docker returns, the relay stops, the directory goes, and the
runtime record is marked ended (on a nonzero exit, after docker lists no
container for it). A run record holds `modelKeyId`, the key's id (12 hex of
its sha256), never the key. The key's exact text is redacted from
`stderr.txt`, the response and the cause line.
The mock adapter takes no key, no proxy and no relay, and is never checked.
`mosaic-task show <runId>` on a run that did not succeed adds one line after
`exit:`, `cause: <reason>`. A reason the runner decided is said in words
(`the run passed its time limit and was stopped`, `the response did not match
the expected text`, `processing was halted during the run`). Otherwise it is
`worker said: <line>`, one line from the run's `stderr.txt` with compose's
progress and warning lines passed over, redacted (paths, links, emails,
key-shaped strings) and withheld when it holds a value from the environment.
With no such line it says the worker container could not be started or the
worker exited with its code and wrote nothing that can be shown. `mosaic-task
run` prints the same line as `cause: <reason>` after `response:` when the run
did not succeed.
A pi worker is judged by pi's final stop reason as well as its exit status
(worker-stop-reason). In print mode pi exits 0 when the model reaches its
output limit, with nothing printed, so that run used to count as succeeded.
The container's entrypoint sets `MOSAIC_PI_FINAL_STOP=1` for a headless pi
run, and the pi adapter then runs pi through `adapters/pi/final-stop.mjs`,
which reads pi's JSON event stream. A finished stop prints the answer as
before. An error or aborted stop fails with `pi stopped on an error: <pi's
error>`, and an output-limit stop fails with `pi stopped with no finished
answer: the model reached its output limit`. The reason stays
`exit-nonzero`, and the line is the `worker said:` cause. Managed sessions
don't set the variable and keep text mode. In the container the wrapper is
process 1 and passes SIGTERM, SIGINT and SIGHUP to pi, so a SIGINT that
reaches the container now stops pi (exit 130). Before, pi was process 1 and
ignored SIGINT.
Legacy worker containers use a typed per-launch mount contract. Task, mission,
user and session-fork snapshots are read-only. A fork source is opened without
following its final symlink, limited to 16 MiB, checked for owner/type and
@@ -65,8 +145,8 @@ addition to its full repository gate; it was not run by this source repair:
```bash
node --test --test-concurrency=1 packages/processing-control/tests/containers.test.mjs packages/processing-control/tests/coordinator.test.mjs packages/processing-control/tests/effects.test.mjs packages/processing-control/tests/mount.test.mjs packages/processing-control/tests/runtimes.test.mjs packages/processing-control/tests/state.test.mjs packages/processing-control/tests/stop-run.test.mjs packages/processing-control/tests/task.test.mjs
node --test scripts/test-darkwing-launch.mjs
scripts/test-task.sh
scripts/build.sh
scripts/test-task.sh
```
## Agent (interactive TUI)
@@ -89,9 +169,9 @@ contract's tools are a ceiling the seat definition or `--tools` may narrow,
never escalate past. Missing/invalid contract refuses the launch; empty
intersection = loud tool-free seat. An explicit `MOSAIC_AGENTS_DIR` override
that cannot resolve the named seat also refuses (#46) — unset the override
for the M13 plain governed TUI. `--auth <account>` injects
`auth.<account>.json` (beside the active credential file) as the launch's
`PI_AUTH_FILE`; a missing/invalid account refuses (M19).
for the M13 plain governed TUI. `--auth` is retired (worker-cutover): it
refuses with exit 4 and `agent: --auth is retired. Workers use the model
proxy; set up accounts on Settings > Model setup.`
For native repository development, opt in with a **leading** `--host-dev`:
@@ -116,21 +196,25 @@ provider; resolution order `--api-key` → `auth.json` → env → models.json).
No credential material is ever printed — provider names, credential types,
and env var NAMES only.
Ownership rule (#48): `~/.pi` is read-only to the stack, permanently. The
only interaction is the existing read-only container mount of the default
credential (`PI_AUTH_FILE`, default `~/.pi/agent/auth.json`). Mosaic-managed
accounts live under the data root: `<dataRoot>/auth/<account>.json`, perms
0600 (mirroring `scripts/gitea-api.sh` hygiene — loose perms are flagged in
listings and refused by `--auth`).
Ownership rule (#48): `~/.pi` is read-only to the stack, permanently. Since
worker-cutover no worker container mounts it: a worker reaches its model
through the model proxy with the worker key Mosaic issues, and accounts are
set up on Settings > Model setup. `auth.sh` still reports the host pi file
(`PI_AUTH_FILE`, default `~/.pi/agent/auth.json`) for host-side pi use, and
lists old files under `<dataRoot>/auth/<account>.json` for cleanup; nothing
launches with them.
| Command | Purpose | Notes |
|---|---|---|
| `scripts/auth.sh status` | Report both credential sources | Default harness credential (read-only) + mosaic-managed accounts; never prints material |
| `scripts/auth.sh accounts` | List mosaic-managed accounts | Under the data root; marks the active one; flags non-0600 |
| `scripts/auth.sh status` | Report the host pi file, the default run and old accounts | Never prints material |
| `scripts/auth.sh accounts` | List old accounts under the data root | Marks the one `PI_AUTH_FILE` names; flags non-0600 |
`agent.sh --auth <account>` injects `<dataRoot>/auth/<account>.json` as the
launch's `PI_AUTH_FILE`; missing/symlinked/non-0600 accounts refuse.
Headless task runs keep the default credential.
`auth.sh status` prints one line for the default run:
```
default run: provider zai (config execution.provider); the worker uses the model proxy with a key Mosaic issues (Settings > Model setup)
default run: provider zai (config execution.provider); the adapter needs no credential
```
## Release
@@ -142,6 +226,28 @@ Headless task runs keep the default credential.
| `scripts/release.sh status` | Release, tag, active pointer, log | Safe on empty state |
| `scripts/release.sh ensure` | Self-determination: align active pointer to `RELEASE` | Fast path restores a missing/mismatched pointer without a gate; slow path packages + health-gates first. Invoked automatically at launch |
A failed health check says which run failed and why before it refuses. With
a run record, the reason is the `cause:` line `mosaic-task show` prints:
```
release: health check run: r-20261120T120000Z-a1b2c3
release: reason: the response did not match the expected text
release: refusing activate: health check failed
```
When the runner gave no run id (it refused before the run started: a launch
check from the table above, a config problem), the reason is the runner's last
refusal line, picked and redacted by `scripts/health-reason.mjs`:
```
release: health check gave no run id
release: reason: proxy-not-configured: The model proxy is not set up. Set it up on Settings > Model setup.
release: refusing activate: health check failed
```
Neither prints a key, an environment value or a host path; the run id is the
only identifier.
## Conductor (worker patches)
```bash
@@ -1379,6 +1485,36 @@ evidence (`proxy-restart-unconfirmed` otherwise). `proxy init-lock
status|clear` clears a stale init lock only when every bus is stopped and no
proxy state exists. Details: `packages/model-proxy/README.md`.
`proxy ride --instance NAME` makes this instance's worker runs use instance
NAME's model proxy (proxy-ride first slice, Jason 2026-11-21). Both instances
belong to the OS user running the command. It writes
`<configDir>/proxy-ride.json` as `{"version":1,"instance":"NAME"}`, mode 0600,
then loads the system config the way a worker run does and removes the file
again on any refusal. `proxy ride --off` removes the file without loading the
config. Both refuse `human-required` in an agent shell or with `--tester`. The
riding instance's `mosaic-config.mjs validate` prints the owner's `proxy` with
`owner: {instance, configDir, dataRoot}`; worker runs (`verify.sh`, `hello.sh`,
`agent.sh`, `run-task.sh`) then use the owner's management key, lock, fence,
state directory and model on/off list. The worker key keeps the riding
instance's own tag and file. Buses, the Console and every other verb load the
config without that proxy, as on an instance with no proxy, and
`MOSAIC_INSTANCE_PROXY_CONFIGURED` stays off, so no proxy unit is installed for
the rider. Every validate resolves the owner, so while `proxy-ride.json`
exists an owner problem refuses every verb on the rider, buses and the Console
included; `proxy ride --off` is the way out. Run the owner with its own config
path (`--instance NAME`, or `MOSAIC_CONFIG` set to exactly
`~/.config/mosaic-NAME/config.json`): the rider locks the owner's proxy state
with that path, and the first side to lock it fixes the lock identity, so the
other side refuses `proxy-lock-invalid`. Refusals (validate exit 2):
`proxy-ride-invalid` (a bad file, a bad name, the instance's own name, or an owner with the same data root),
`proxy-ride-owner-missing` (the named
instance has no valid config, claim or `proxy.json`, or rides another instance
itself), `proxy-ride-conflict` (`proxy.json` and `proxy-ride.json` both exist;
`proxy init` also refuses it on a riding instance). Each prints one line with
the owner's next step, except under `run-task.sh`, where a ride refusal still
shows as a JSON parse error, as every config refusal there does. `--off` leaves the rider's worker key in the owner's
proxy; it is not removed yet.
## Agent limits (`scripts/mosaic limits`)
View: `scripts/mosaic limits --business ID --json`. Set: `scripts/mosaic limits
+1 -1
View File
@@ -9,7 +9,7 @@
"version": "0.0.3",
"license": "UNLICENSED",
"dependencies": {
"@earendil-works/pi-coding-agent": "0.85.1",
"@earendil-works/pi-coding-agent": ">=0.85.1",
"oauth4webapi": "3.8.8",
"yaml": "2.9.0"
}
+1 -1
View File
@@ -5,7 +5,7 @@
"description": "Minimal Mosaic Stack container proof of concept: one Pi agent, four local contract files, one real model request returning MOSAIC_HELLO_OK.",
"license": "UNLICENSED",
"dependencies": {
"@earendil-works/pi-coding-agent": "0.85.1",
"@earendil-works/pi-coding-agent": ">=0.85.1",
"oauth4webapi": "3.8.8",
"yaml": "2.9.0"
}
+135 -1
View File
@@ -554,6 +554,62 @@ const INSTALLER_COMMENT = 'mosaic-installer-';
// business ID_PATTERN, which starts with [a-z0-9], so no instance can be named this.
const NOTICE_ROLE = '_bus';
const NOTICE_RUN = 'bus';
// #1873 chat-fail F2: the one plain line a refused thread start, or a run the model proxy stopped
// for silence (proxy-stall), leaves in the conversation for each message it left unanswered
// (failureNotice below). It is here, not in a file of its own, because the runner imports this
// file in the session sandbox, which mounts only RUNNER_FILES (packages/seat/src/sandbox.mjs).
//
// The line has the form a failed turn's reply has (packages/harness/src/failure-notice.mjs), so the
// chat renders it the same way: "The turn failed: <cause and next step>\nDetails: <code>". The
// text is fixed per code. A refusal's own reason is never put in it: it can name paths, models
// and accounts. Codes are the launcher's existing refusal codes and the existing proxy-stall stop
// reason; none is added here.
const MODEL = "the model this agent uses isn't available from the model service now. Pick another model, or check Settings > Model setup.";
const PROXY = "the model proxy isn't ready. Check Settings > Model setup.";
const ACCOUNT = "the agent's model account isn't signed in or set up. Sign in again, or check the account in Settings > Model setup.";
const LIMIT = "the agent reached a usage limit. Wait for it to reset, or check the limits in Settings.";
const SETUP = "the agent's setup can't start now. Check the agent in Agents.";
const OTHER = "the assistant couldn't start. Try again later, or check the agent in Agents.";
const LINES = new Map([
['proxy-stall', "the model service gave no answer in time, so the turn was stopped. Try again, or check the account in Settings > Model setup."],
['model-provider-unavailable', MODEL],
['model-catalogue-unavailable', MODEL],
['model-harness-incompatible', MODEL],
['proxy-uncertain', PROXY],
['no-session-credential', PROXY],
['account-credential-missing', ACCOUNT],
['account-credential-ambiguous', ACCOUNT],
['account-is-credential', ACCOUNT],
['credential-unavailable', ACCOUNT],
['limit-reached', LIMIT],
['limit-check-failed', LIMIT],
['harness-off', SETUP],
['harness-refused', SETUP],
['harness-failed', SETUP],
['unsupported-harness', SETUP],
['thinking-unsupported', SETUP],
['plugin-disabled', SETUP],
['skill-not-enabled', SETUP],
['skill-refused', SETUP],
['no-role-contract', SETUP],
['definition-holds-credential', SETUP],
['project-archived', "this project is archived, so the assistant can't reply here."],
]);
// A start that waits for a free place, a run that is still ending, a held lock, a lost race, a
// project still being set up or an operator's pause starts by itself once that clears: no line.
const WAITS = new Set(['capacity-full', 'instance-running', 'legacy-run-live', 'session-workspace-busy',
'processing-generation-changed', 'proxy-lock-busy', 'project-pending', 'processing-halted']);
const NOTICE_CODE = /^[a-z][a-z-]{0,63}$/;
// The body for a code, or null when the code leaves no line.
function failureNoticeBody(code) {
if (typeof code !== 'string' || !NOTICE_CODE.test(code) || WAITS.has(code)) return null;
const line = LINES.get(code) ?? (code.startsWith('proxy-') ? PROXY : OTHER);
return `The turn failed: ${line}\nDetails: ${code === 'proxy-stall' ? 'stopped' : 'start refused'}: ${code}`;
}
// Row 143 (#1637, REQ-PROJ-4): a human save of this file through project.file.save sends one notice to each
// instance of these definitions. Their answer goes to the instance of SPEC_ANSWER_TO, the chief of staff,
// because only the user and the chief of staff may send to the assistant (REQ-ASSIST-2).
@@ -652,6 +708,7 @@ export class Broker {
#credentialVerbs;
#proxyInit;
#proxyFenceClear;
#proxyTriageKey;
#proxyPage;
#project;
#library;
@@ -716,7 +773,7 @@ export class Broker {
#processing;
#taskTracker;
// credentialStatus is the runtime's metadata-only Credentials.status(); the broker never gets use().
constructor({ store, businesses, pluginService = null, secretCheck = () => {}, credentialStatus = () => [], organization = null, hires = null, autonomy = null, project = null, library = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, proxyInit = null, proxyFenceClear = null, proxyPage = null, hostGrant = null, hostKeys = null, processing = null, testerOn = null, grantScope = null, roleFile = null, giteaGrant = null, copyGrant = null, copyLaunch = null, copyRevoker = null, giteaKeys = null, access = null, accessStamp = null, compactions = null, requestLifecycle = null, requirementsHost = null, workspaceRead = null, taskTracker = null }) {
constructor({ store, businesses, pluginService = null, secretCheck = () => {}, credentialStatus = () => [], organization = null, hires = null, autonomy = null, project = null, library = null, usage = null, limits = null, harnessOn = null, credentialVerbs = null, proxyInit = null, proxyFenceClear = null, proxyTriageKey = null, proxyPage = null, hostGrant = null, hostKeys = null, processing = null, testerOn = null, grantScope = null, roleFile = null, giteaGrant = null, copyGrant = null, copyLaunch = null, copyRevoker = null, giteaKeys = null, access = null, accessStamp = null, compactions = null, requestLifecycle = null, requirementsHost = null, workspaceRead = null, taskTracker = null }) {
if (!object(businesses) || !Object.keys(businesses).length) fail('invalid-business');
// Row 194 b15 (sage-2501): the function-argument checks come before anything reads the store.
if (harnessOn !== null && typeof harnessOn !== 'function') throw new TypeError('harnessOn must be a function');
@@ -746,6 +803,7 @@ export class Broker {
this.#credentialVerbs = credentialVerbs;
this.#proxyInit = proxyInit;
this.#proxyFenceClear = proxyFenceClear;
this.#proxyTriageKey = proxyTriageKey;
this.#proxyPage = proxyPage;
this.#hostGrant = hostGrant;
this.#hostKeys = hostKeys;
@@ -2006,6 +2064,61 @@ export class Broker {
this.#launcherReleased.add(key);
return { released: true };
}
// #1873 chat-fail F2: a thread start the launcher refused, or a run the model proxy stopped for
// silence (proxy-stall), leaves the person who wrote no answer. This puts one plain line in the
// conversation for each message from the human, a user or the tester it left unanswered, as a
// RESULT in reply to it, the way #landMessage reports a refused landing. A message in a thread is
// to or from the thread's instance (trigger message_threads_match), so the line is from the
// instance, with from_run NOTICE_RUN: no run sent it, the bus did, and a failed turn's own line is
// the instance's too. With { thread, session } the messages are the session's pending ones, for a
// refused start; with { run } they are the ones delivered to that run and never read, for a
// stopped run. A message the instance has already answered gets no line, so a refusal the
// launcher meets on every tick leaves one line per message. The text is fixed per code
// (failureNoticeBody above). Trusted launcher API, deliberately NOT a socket verb.
failureNotice(record) {
keys(record, ['business', 'code', 'thread', 'session', 'run'], ['business', 'code']);
this.#business(identifier(record.business));
if (typeof record.code !== 'string' || !NOTICE_CODE.test(record.code)) fail('invalid-request');
const byRun = record.run !== undefined;
if (byRun === (record.thread !== undefined || record.session !== undefined)) fail('invalid-request');
let session = null, instance, where, param;
if (byRun) {
const run = this.#runs.get(record.business + ':' + identifier(record.run)) ?? fail('unknown-run');
session = run.session ?? null;
instance = run.role;
where = `EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op='delivered' AND d.holder_run=?)
AND NOT EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op='read')`;
param = run.run;
} else {
session = identifier(record.session);
const se = this.#store.get('SELECT instance FROM sessions WHERE id=? AND thread=? AND business=?', session, identifier(record.thread), record.business) ?? fail('session-not-found');
instance = se.instance;
where = `NOT EXISTS(SELECT 1 FROM deliveries d WHERE d.message=m.id AND d.op IN ('delivered','read'))`;
param = null;
}
const body = failureNoticeBody(record.code);
if (body === null) return { posted: 0 };
const scope = session ? 'm.id IN (SELECT message FROM message_threads WHERE session=?)' : 'NOT EXISTS(SELECT 1 FROM message_threads mt WHERE mt.message=m.id)';
return this.#store.transaction(() => {
const open = this.#store.all(
`SELECT m.id, m.from_role FROM messages m WHERE m.business=? AND m.to_role=? AND ${scope}
AND m.from_role IN ('human','user','tester') AND ${where}
AND NOT EXISTS(SELECT 1 FROM message_cancels x WHERE x.message=m.id)
AND NOT EXISTS(SELECT 1 FROM messages n WHERE n.in_reply_to=m.id AND n.from_role=m.to_role)
ORDER BY m.seq`,
...[record.business, instance, ...(session ? [session] : []), ...(param === null ? [] : [param])],
);
for (const m of open) {
const mid = randomUUID();
this.#store.run(
'INSERT INTO messages(id,at,business,from_role,from_run,to_role,class,in_reply_to,decision,corrects,body) VALUES(?,?,?,?,?,?,?,?,?,?,?)',
mid, this.#now(), record.business, instance, NOTICE_RUN, m.from_role, 'RESULT', m.id, null, null, body,
);
this.#route(record.business, mid, m.from_role, session);
}
return { posted: open.length };
});
}
// Row 271 (264d, #1776, D7 steps 3-5): the business's clears still in the launcher's hands, as of
// each op's latest row. An open op waits for its drained run's stop and exit proof, then its
// retirement. A retired op whose successor hasn't started, and wasn't cancelled, waits for its
@@ -6109,6 +6222,27 @@ export class Broker {
return result;
}).catch(error => { throw this.#refused(s, new BusError(/^proxy-[a-z-]+$/.test(error.code ?? '') ? error.code : 'proxy-down'), { verb: request.verb }); });
}
// proxy-scope slice 4 (Jason 2026-11-19): ensure or revoke the instance's Triage key through the
// proxy this bus rides. Owner only, before every other gate, like proxy.fence.clear. The
// answer, the event and the refusal carry key ids and codes only, never a key.
if (request.verb === 'proxy.triage.key') {
this.#human(s);
if (!this.#proxyTriageKey) fail('not-configured');
const args = request.args ?? {};
keys(args, ['op'], ['op']);
if (args.op !== 'ensure' && args.op !== 'revoke') fail('invalid-request');
const ID = /^[0-9a-f]{12}$/;
const idList = v => Array.isArray(v) && v.every(id => typeof id === 'string' && ID.test(id));
return Promise.resolve().then(() => this.#proxyTriageKey(args.op)).then(result => {
const ACTIONS = args.op === 'ensure' ? ['issued', 'held', 'replaced'] : ['revoked', 'none'];
if (!object(result) || !ACTIONS.includes(result.action) || !idList(result.addedKeyIds) || !idList(result.removedKeyIds)
|| (args.op === 'ensure' && !(typeof result.keyId === 'string' && ID.test(result.keyId)))) fail('proxy-client-response');
const answer = { op: args.op, action: result.action, addedKeyIds: [...result.addedKeyIds], removedKeyIds: [...result.removedKeyIds],
...(args.op === 'ensure' ? { keyId: result.keyId, rotateBy: /^\d{4}-\d{2}-\d{2}$/.test(result.rotateBy ?? '') ? result.rotateBy : null } : {}) };
this.#store.transaction(() => this.#event(s, 'action.allowed', { call: 'proxy.triage.key', ...answer }, null));
return answer;
}).catch(error => { throw this.#refused(s, error instanceof BusError ? error : new BusError('storage-unavailable', 'triage-key-failed'), { verb: request.verb }); });
}
// FED R3 (design 3.5, 3.7 trigger 1): a user reaches the collaborator list only, and each
// request reads the access file again, so a removed or expired grant refuses the next one.
let grants = null;
+12 -6
View File
@@ -1,10 +1,15 @@
// In-memory only; a new loader is made for every refresh so disabled or refused
// modules can never inherit a loader's last-good authority.
import { createLoader } from '../../plugins/src/loader.mjs';
import { readOffered } from '../../model-proxy/src/models-offered.mjs';
const validId = s => typeof s === 'string' && /^[a-z0-9][a-z0-9._-]{0,63}$/.test(s);
const validModel = s => typeof s === 'string' && /^[A-Za-z0-9][A-Za-z0-9._:/-]{0,199}$/.test(s);
const disabled = () => ({ status: 'disabled', observedAt: null, models: [], notices: [] });
export function createModelCatalogue({ client, settings, pluginDataRoot = null, now = Date.now, makeLoader = createLoader, proxyLock = null }) {
// #1873 models-offered: the owner's on/off file, read on every catalogue read so a Console write
// counts at the next read. No data root (tests, a fixture bus) means no file: every model stays on.
const offeredNow = () => pluginDataRoot ? readOffered(pluginDataRoot) : { state: 'absent', offered: null, revision: null };
const offeredMark = o => o.state === 'damaged' ? 'damaged' : o.revision;
const cache = new Map(), flights = new Map(); let generation = 0, writing = 0;
function invalidate() { generation++; cache.clear(); }
function begin() { writing++; invalidate(); }
@@ -15,7 +20,7 @@ export function createModelCatalogue({ client, settings, pluginDataRoot = null,
if (!Array.isArray(names) || names.length > 32 || names.some(n => !validId(n)) || new Set(names).size !== names.length) throw Error();
return false;
}
async function refresh(id, previous) {
async function refresh(id, previous, offered) {
try {
if (off(id)) return disabled();
const names = [...settings(id)['model-provider']];
@@ -27,11 +32,11 @@ export function createModelCatalogue({ client, settings, pluginDataRoot = null,
const c = client();
const proxy = method => async (...args) => { if (!c) throw Error('model proxy not configured'); return c[method](...args); };
const answers = await Promise.all(loader.snapshot('model-provider').map(async ({ name, module }) => {
const answer = await module.catalogue({ listAccounts: proxy('getCredentials'), listModels: proxy('getCredentialModels'), listGroups: proxy('getUpstreamGroups'), now: () => new Date(now()).toISOString() });
const answer = await module.catalogue({ listAccounts: proxy('getCredentials'), listModels: proxy('getCredentialModels'), listGroups: proxy('getUpstreamGroups'), now: () => new Date(now()).toISOString(), offered });
if (answer?.status === 'unavailable' && answer.reason === 'model-key-collision') throw Object.assign(Error(), { code: 'model-key-collision' });
if (answer?.status !== 'ready' || !Array.isArray(answer.models) || answer.models.length > 10000 || !Array.isArray(answer.notices) || answer.notices.length > 100) throw Error();
for (const m of answer.models) {
if (!validId(m.key) || m.module !== name || !validId(m.upstream) || !validId(m.provider) || !validModel(m.model) || typeof m.label !== 'string' || m.label.length > 300 || /[\x00-\x1f\x7f]/.test(m.label) || typeof m.available !== 'boolean' || ![null, 'unverified'].includes(m.reason) || !Array.isArray(m.harnesses) || m.harnesses.length > 32 || m.harnesses.some(h => !validId(h))) throw Error();
if (!validId(m.key) || m.module !== name || !validId(m.upstream) || !validId(m.provider) || !validModel(m.model) || typeof m.label !== 'string' || m.label.length > 300 || /[\x00-\x1f\x7f]/.test(m.label) || typeof m.available !== 'boolean' || ![null, 'unverified', 'model-off'].includes(m.reason) || (m.isNew !== undefined && m.isNew !== true) || !Array.isArray(m.harnesses) || m.harnesses.length > 32 || m.harnesses.some(h => !validId(h))) throw Error();
}
if (answer.notices.some(n => typeof n !== 'string' || n.length > 300 || /[\x00-\x1f\x7f]/.test(n))) throw Error();
return answer;
@@ -50,14 +55,15 @@ export function createModelCatalogue({ client, settings, pluginDataRoot = null,
if (writing) return unavailable();
try { if (off(id)) { cache.delete(id); return disabled(); } } catch { return { status: 'unavailable', observedAt: null, models: [], notices: [] }; }
const existing = cache.get(id);
const fingerprint = JSON.stringify([settings(id)['model-provider'], sharedGeneration]);
const offered = offeredNow();
const fingerprint = JSON.stringify([settings(id)['model-provider'], sharedGeneration, offeredMark(offered)]);
if (!fresh && existing && existing.fingerprint === fingerprint && now() - existing.time < 30000) return structuredClone(existing.value);
const flightKey = `${id}:${fingerprint}:${generation}`;
let flight = flights.get(flightKey);
if (!flight || fresh) {
const at = generation;
flight = refresh(id, existing?.value).then(value => {
if (generation !== at || JSON.stringify([settings(id)?.['model-provider'], proxyLock?.generation()]) !== fingerprint) return null;
flight = refresh(id, existing?.value, offered.offered).then(value => {
if (generation !== at || JSON.stringify([settings(id)?.['model-provider'], proxyLock?.generation(), offeredMark(offeredNow())]) !== fingerprint) return null;
if (cache.size >= 128 && !cache.has(id)) cache.delete(cache.keys().next().value);
cache.set(id, { time: now(), fingerprint, value }); return value;
}).catch(() => null);
+3
View File
@@ -28,6 +28,7 @@ const LAUNCH_OPS = new Set([
'landHeld',
'threadsState',
'threadIdle',
'failureNotice',
'mayLaunch',
'landHolders',
'landRelease',
@@ -67,6 +68,8 @@ function launchOp(m) {
// Row 121 (#1615): the launcher's view of the business's threads, and its idle release.
if (m.op === 'threadsState') return runtime.threadsState(m.business);
if (m.op === 'threadIdle') return runtime.threadIdle(m.record);
// #1873 chat-fail F2: one plain line per message a refused thread start or a proxy-stall stop left unanswered.
if (m.op === 'failureNotice') return runtime.failureNotice(m.record);
// Row 125 (#1619, REQ-LIMIT-4): the limit checks' answer before a launch (async).
if (m.op === 'mayLaunch') return runtime.mayLaunch(m);
// Row 156 (#1651): the launcher's landing pass: who holds a merge decision, and the stop of a
+4 -2
View File
@@ -1,6 +1,8 @@
// Pure broker/runner parsing vocabulary. No host management or state imports.
export const PROXY_PAGE_VERBS = Object.freeze(['proxy.accounts', 'proxy.login.start', 'proxy.login.submit',
'proxy.login.status', 'proxy.login.cancel', 'proxy.apikey.add', 'proxy.priority.set', 'proxy.group.models.set', 'proxy.keys']);
'proxy.login.status', 'proxy.login.cancel', 'proxy.apikey.add', 'proxy.priority.set', 'proxy.group.models.set', 'proxy.keys',
// #1873 account-remove (Jason 2026-11-20): one account out of the model service, confirmed.
'proxy.account.remove']);
export const PROXY_PAGE_CODES = Object.freeze(['not-configured', 'human-required', 'provider-not-allowed',
'login-state-unknown', 'callback-url-invalid', 'apikey-invalid', 'priority-invalid', 'priority-unconfirmed',
'priority-write-refused', 'models-invalid', 'models-ambiguous', 'models-write-refused', 'models-unconfirmed', 'proxy-uncertain', 'proxy-down', 'invalid-request']);
'priority-write-refused', 'models-invalid', 'models-ambiguous', 'models-write-refused', 'models-unconfirmed', 'remove-write-refused', 'remove-unconfirmed', 'proxy-uncertain', 'proxy-down', 'invalid-request']);
+99 -20
View File
@@ -3,7 +3,7 @@ import { randomBytes } from 'node:crypto';
import { isDeepStrictEqual } from 'node:util';
import { sha256, refuse } from '../../model-proxy/src/common.mjs';
import { PROXY_PAGE_CODES } from './proxy-page-vocabulary.mjs';
import { ZAI_URL, MODEL_ID } from '../../plugins/model-provider/cliproxy/index.mjs';
import { MODEL_ID, PROVIDERS, publicProviders } from '../../plugins/model-provider/cliproxy/index.mjs';
export { PROXY_PAGE_VERBS, PROXY_PAGE_CODES } from './proxy-page-vocabulary.mjs';
const object = x => x !== null && typeof x === 'object' && !Array.isArray(x);
function fields(args, names) {
@@ -13,19 +13,61 @@ const priorityOk = n => Number.isInteger(n) && n >= -1000 && n <= 1000;
const identifier = x => typeof x === 'string' && x.length > 0 && x.length <= 512;
const stateOk = x => typeof x === 'string' && /^[a-zA-Z0-9_-]{1,512}$/.test(x);
const keyId = value => sha256(value).slice(0, 12);
const providerName = provider => ({ claude: 'Claude', codex: 'Codex', 'z.ai': 'z.ai', other: 'Other' })[provider];
// #1873 settings-backend-a3: provider names, the login allow list and the key allow list all come
// from the plugin's PROVIDERS. An account the list does not name reads Other.
// An entry counts only with what its door needs: a sign-in host, or a key base URL.
const usable = p => p.signIn === 'oauth' ? typeof p.signInHost === 'string' : p.signIn === 'apikey' && typeof p.baseUrl === 'string';
const listed = (providers, id, signIn) => providers.find(p => p.id === id && p.signIn === signIn && usable(p)) ?? null;
const providerName = (providers, provider) => providers.find(p => p.id === provider)?.label ?? 'Other';
// Credential labels/emails and group names are arbitrary upstream strings. A stable
// generated display name provides identification without reflecting a misplaced token.
function view(id, provider, status, priority) {
return { accountId: keyId(id), provider, displayName: `${providerName(provider)} account ${keyId(id)}`,
function view(providers, id, provider, status, priority) {
return { accountId: keyId(id), provider, displayName: `${providerName(providers, provider)} account ${keyId(id)}`,
status, priority: priorityOk(priority) ? priority : 0 };
}
// #1873 chat-fail F3: one plain line under an account the proxy reports as not working, with
// the next step. It is built only from the structured fields of the account read this page
// already makes. status_message is matched against fixed patterns and never shown; a label,
// email, recovery hint or refresh error message is never read into the line.
const SIGN_IN = 'The account needs to sign in again. Sign in again, or add another account.';
const NO_REASON = 'The model service gave no reason. Wait and try again later, sign in again, or add another account.';
const USAGE_TEXT = /usage limit|quota|rate.?limit|limit reached|too many requests|\b429\b|insufficient.?balance|credit balance/i;
// A bare "expired" is not enough: a usage window expires too.
const SIGN_IN_TEXT = /\b401\b|unauthori[sz]ed|invalid.?grant|\b(?:\w*[_-])?(?:tokens?|sessions?|credentials?|logins?|auth\w*|grants?)\s+(?:(?:has|have|is|are|was|were)\s+)?(?:been\s+)?expired|\brevoked\b|\bsign.?in\b|\b(?:re-?)?log.?in\b|\bre-?auth/i;
const RFC3339 = /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})(?:\.(\d{1,9}))?(Z|[+-]\d{2}:\d{2})$/;
// A reset time the read gives, as "YYYY-MM-DD HH:MM UTC" made from the parsed date, only when
// it is a valid RFC 3339 time still ahead of now. Anything else gives no time.
function resetTime(value, now) {
const m = typeof value === 'string' ? RFC3339.exec(value) : null;
if (!m || +m[2] < 1 || +m[2] > 12 || +m[3] < 1 || +m[3] > 31 || +m[4] > 23 || +m[5] > 59 || +m[6] > 59) return null;
// A day the month doesn't have (02-31) would roll into the next month, so it is refused.
const day = new Date(Date.UTC(2000, +m[2] - 1, +m[3]));
day.setUTCFullYear(+m[1]);
if (day.getUTCMonth() !== +m[2] - 1 || day.getUTCDate() !== +m[3]) return null;
const at = Date.parse(`${m[1]}-${m[2]}-${m[3]}T${m[4]}:${m[5]}:${m[6]}${m[8]}`);
if (!Number.isFinite(at) || at <= now) return null;
return `${new Date(at).toISOString().slice(0, 16).replace('T', ' ')} UTC`;
}
function accountReason(row, now) {
const message = typeof row.status_message === 'string' ? row.status_message.slice(0, 4096) : '';
const time = resetTime(row.next_retry_after, now);
const usage = time ? `The account is out of usage. It resets at ${time}. Wait until then, or add another account.`
: 'The account is out of usage. Wait until it resets, or add another account.';
// The structured flags come first, then the message patterns.
if (row.requires_sign_in === true || (object(row.refresh_error) && Object.keys(row.refresh_error).length > 0)) return SIGN_IN;
if (row.quota_blocked === true) return usage;
if (SIGN_IN_TEXT.test(message)) return SIGN_IN;
if (USAGE_TEXT.test(message)) return usage;
if (time) return `The model service paused this account until ${time}. Wait until then, sign in again, or add another account.`;
return NO_REASON;
}
export class ProxyPage {
#client; #tag; #records; #lock; #org; #dirty = false; #now; #timeout; #onChange; #pending = new Map(); #writes = Promise.resolve();
constructor({ client, instanceTag, keyRecords = () => [], now = Date.now, loginTimeoutSeconds = 300, onChange = () => {}, lock = null, org = null }) {
#client; #tag; #records; #lock; #org; #dirty = false; #now; #timeout; #onChange; #providers; #pending = new Map(); #writes = Promise.resolve();
// providers: the plugin's PROVIDERS; a test passes a copy with one taken out.
constructor({ client, instanceTag, keyRecords = () => [], now = Date.now, loginTimeoutSeconds = 300, onChange = () => {}, lock = null, org = null, providers = PROVIDERS }) {
if (!client || !/^[a-zA-Z0-9_-]{1,64}$/.test(instanceTag) || typeof keyRecords !== 'function' || typeof now !== 'function'
|| typeof onChange !== 'function' || !Number.isSafeInteger(loginTimeoutSeconds) || loginTimeoutSeconds <= 0) throw new TypeError('invalid proxy page settings');
this.#lock = lock; this.#org = org;
|| typeof onChange !== 'function' || !Number.isSafeInteger(loginTimeoutSeconds) || loginTimeoutSeconds <= 0 || !Array.isArray(providers)) throw new TypeError('invalid proxy page settings');
this.#lock = lock; this.#org = org; this.#providers = providers;
this.#client = client; this.#tag = instanceTag; this.#records = keyRecords; this.#now = now; this.#timeout = loginTimeoutSeconds * 1000; this.#onChange = onChange;
}
// Serializes read-modify-write provider config within the bus. P2 owns the
@@ -48,28 +90,30 @@ export class ProxyPage {
const rows = [];
for (const row of response.files) {
if (!identifier(row?.id) || !identifier(row?.name)) continue;
const provider = ['claude', 'codex'].includes(row.provider) ? row.provider : 'other';
const provider = listed(this.#providers, row.provider, 'oauth') ? row.provider : 'other';
const status = row.disabled === true ? 'disabled' : row.unavailable === true ? 'unavailable'
: ['ready', 'active', 'error', 'disabled', 'unavailable'].includes(row.status) ? row.status : 'unknown';
rows.push({ ...view(row.id, provider, status, row.priority), source: 'oauth', name: row.name });
rows.push({ ...view(this.#providers, row.id, provider, status, row.priority), source: 'oauth', name: row.name,
...(['unavailable', 'error'].includes(status) ? { reason: accountReason(row, this.#now()) } : {}) });
}
for (const [groupIndex, group] of groups.entries()) {
if (!Array.isArray(group?.keys)) continue;
for (const [keyIndex, key] of group.keys.entries()) {
// auth_index is generated by the proxy. Never fall back to hashing api-key.
if (!identifier(key?.auth_index)) continue;
const provider = group['base-url'] === ZAI_URL ? 'z.ai' : 'other';
const provider = this.#providers.find(p => p.signIn === 'apikey' && usable(p) && p.baseUrl === group['base-url'])?.id ?? 'other';
const status = key.disabled === true || group.disabled === true ? 'disabled' : 'active';
const configured = group.models ?? [];
if (!Array.isArray(configured) || configured.length > 32 || configured.some(m => !m || typeof (m.alias || m.name) !== 'string' || !MODEL_ID.test(m.alias || m.name))) refuse('proxy-down');
rows.push({ ...view(key.auth_index, provider, status, key.priority ?? group.priority), source: 'apikey', groupIndex, keyIndex, models: configured.map(m => m.alias || m.name) });
rows.push({ ...view(this.#providers, key.auth_index, provider, status, key.priority ?? group.priority), source: 'apikey', groupIndex, keyIndex, models: configured.map(m => m.alias || m.name) });
}
}
return { rows, groups };
return { rows, groups, credentialNames: response.files.map(row => row?.name).filter(identifier) };
}
#public(row) {
const { accountId, provider, displayName, status, priority } = row;
return { accountId, provider, displayName, status, priority, ...(row.source === 'apikey' && provider === 'z.ai' ? { models: row.models } : {}) };
return { accountId, provider, displayName, status, priority, ...(row.source === 'apikey' && listed(this.#providers, provider, 'apikey') ? { models: row.models } : {}),
...(typeof row.reason === 'string' ? { reason: row.reason } : {}) };
}
#session(state, scope, { reportTimeout = false } = {}) {
const entry = stateOk(state) ? this.#pending.get(state) : null;
@@ -93,17 +137,18 @@ export class ProxyPage {
async #request(verb, args, scope) {
if (verb === 'proxy.accounts') {
fields(args, []);
return { accounts: (await this.#accounts()).rows.map(row => this.#public(row)) };
// The provider list rides along for the page's select: id, label and sign-in kind only.
return { accounts: (await this.#accounts()).rows.map(row => this.#public(row)), providers: publicProviders(this.#providers) };
}
if (verb === 'proxy.login.start') {
fields(args, ['provider']);
if (!['claude', 'codex'].includes(args.provider)) refuse('provider-not-allowed');
const oauth = listed(this.#providers, args.provider, 'oauth');
if (!oauth) refuse('provider-not-allowed');
for (const [state, value] of this.#pending) if (value.expires <= this.#now()) this.#pending.delete(state);
if (this.#pending.size >= 128) refuse('proxy-down');
const answer = await (this.#lock ? this.#write(() => this.#client.oauthStart(args.provider)) : this.#client.oauthStart(args.provider));
let url; try { url = new URL(answer.url); } catch { refuse('proxy-down'); }
const host = args.provider === 'claude' ? 'claude.ai' : 'auth.openai.com';
if (url.protocol !== 'https:' || url.hostname !== host || url.username || url.password || url.port
if (url.protocol !== 'https:' || typeof oauth.signInHost !== 'string' || url.hostname !== oauth.signInHost || url.username || url.password || url.port
|| url.href.length > 8192 || !stateOk(answer.state) || url.searchParams.getAll('state').length !== 1
|| url.searchParams.get('state') !== answer.state || this.#pending.has(answer.state)) refuse('proxy-down');
this.#pending.set(answer.state, { business: scope.business, human: scope.human, expires: this.#now() + this.#timeout });
@@ -148,12 +193,13 @@ export class ProxyPage {
}
if (verb === 'proxy.apikey.add') {
fields(args, ['provider', 'key']);
if (args.provider !== 'z.ai') refuse('provider-not-allowed');
const keyed = listed(this.#providers, args.provider, 'apikey');
if (!keyed) refuse('provider-not-allowed');
if (typeof args.key !== 'string' || !/^[\x21-\x7e]{1,4096}$/.test(args.key)) refuse('apikey-invalid');
return this.#write(async () => {
const groups = await this.#client.getUpstreamGroups();
if (!Array.isArray(groups)) refuse('proxy-down');
const group = { name: `mosaic-zai-${randomBytes(6).toString('hex')}`, 'base-url': ZAI_URL,
const group = { name: `mosaic-${keyed.id.replace(/[^a-z0-9]/g, '')}-${randomBytes(6).toString('hex')}`, 'base-url': keyed.baseUrl,
priority: 0, keys: [{ 'api-key': args.key }] };
if (this.#lock && !isDeepStrictEqual(await this.#client.getUpstreamGroups(), groups)) refuse('proxy-down');
this.#beforeMutation('proxy.apikey.add');
@@ -217,6 +263,39 @@ export class ProxyPage {
return { accountId: row.accountId, priority: args.priority };
});
}
// #1873 account-remove (Jason 2026-11-20): one account out of the model service. A signed-in
// account's credential is deleted; an API key comes out of its key group, and a group left with
// no key comes out whole. Then the accounts are read again: a write the proxy refused is
// remove-write-refused, an account still listed after it is remove-unconfirmed.
if (verb === 'proxy.account.remove') {
fields(args, ['accountId', 'confirm']);
if (typeof args.accountId !== 'string' || !/^[a-f0-9]{12}$/.test(args.accountId) || args.confirm !== true) refuse('invalid-request');
return this.#write(async () => {
const { rows, groups, credentialNames } = await this.#accounts();
const original = this.#lock ? structuredClone(groups) : null;
const matches = rows.filter(row => row.accountId === args.accountId);
if (matches.length !== 1) refuse('invalid-request');
const row = matches[0];
// The DELETE names a credential file; another listed account on the same name would go too.
if (row.source === 'oauth' && credentialNames.filter(name => name === row.name).length !== 1) refuse('invalid-request');
if (this.#lock && !isDeepStrictEqual(await this.#client.getUpstreamGroups(), original)) refuse('proxy-down');
this.#beforeMutation('proxy.account.remove');
try {
if (row.source === 'oauth') await this.#client.deleteCredential(row.name);
else {
const group = groups[row.groupIndex];
const keys = group.keys.filter((_, index) => index !== row.keyIndex);
await this.#client.putUpstreamGroups(keys.length > 0 ? groups.map((g, index) => index === row.groupIndex ? { ...g, keys } : g)
: groups.filter((_, index) => index !== row.groupIndex));
}
} catch (error) {
refuse(error.code === 'proxy-client-refused' || error.code === 'proxy-client-redirect' ? 'remove-write-refused' : 'proxy-down');
}
if ((await this.#accounts()).rows.some(r => r.accountId === args.accountId)) refuse('remove-unconfirmed');
this.#onChange('invalidate');
return { accountId: row.accountId, provider: row.provider, removed: true };
});
}
if (verb === 'proxy.keys') {
fields(args, []);
const values = await this.#client.getKeys();
+12 -1
View File
@@ -71,7 +71,7 @@ group(
"Read current session/status and ask the launcher or owner to finish the supported lifecycle transition. A stale or ended run must not reclaim another holder's identity."
);
group(
"apikey-invalid callback-url-invalid harness-off harness-refused key-quota-no-model-key login-state-unknown model-catalogue-unavailable model-harness-incompatible model-key-collision model-provider-unavailable models-ambiguous models-invalid models-unconfirmed models-write-refused priority-invalid priority-unconfirmed priority-write-refused provider-not-allowed proxy-config-invalid proxy-down proxy-first-output-invalid proxy-init-failed proxy-key-invalid proxy-key-issue-failed proxy-key-refused proxy-key-revoking proxy-open proxy-revoke-revision-conflict proxy-rotate-grace-invalid proxy-uncertain thinking-unsupported",
"apikey-invalid callback-url-invalid harness-off harness-refused key-quota-no-model-key login-state-unknown model-catalogue-unavailable model-harness-incompatible model-key-collision model-provider-unavailable models-ambiguous models-invalid models-unconfirmed models-write-refused priority-invalid priority-unconfirmed priority-write-refused provider-not-allowed proxy-config-invalid proxy-down proxy-first-output-invalid proxy-init-failed proxy-key-invalid proxy-key-issue-failed proxy-key-refused proxy-key-revoking proxy-open proxy-revoke-revision-conflict proxy-rotate-grace-invalid proxy-uncertain remove-unconfirmed remove-write-refused thinking-unsupported",
"R1/R3: owner-managed model and provider configuration",
"Ask the owner to inspect the provider configuration through its supported controls; never put keys or callback contents in messages."
);
@@ -83,10 +83,21 @@ group("proxy-init-busy proxy-init-lock-held proxy-init-lock-absent", "R4: init l
group("proxy-key-transport", "R1/R6: a pasted key travels only over loopback or HTTPS", "Use the generated key, which never leaves the host, or open the Console over HTTPS or on this machine to paste one.");
group("proxy-restart-required proxy-restart-unconfirmed proxy-lock-uncertain", "R4/R6: fence clears only after a confirmed proxy restart", "Restart the proxy from the Proxy page or with mosaic proxy fence clear --restart, so the new process is confirmed on the proxy port, then clear the fence. A fence file with no readable time cannot be proven against: restart the proxy service yourself and run mosaic proxy fence clear --proxy-restarted.");
group("proxy-admin-busy", "R4: one proxy setup action at a time", "Wait for the running fetch, init or unit action to finish (the Proxy page status shows it), then retry.");
// #1873 models-offered (A9, Jason 2026-11-20): the owner's per-provider model lists on the Model setup page.
group("models-offered-conflict models-in-use", "R1/R4: the owner's offered models change one saved list at a time, and never silently under a running organization", "Reload the model list and make the change again. To switch off a model in use, confirm it in place, then give each organization and agent named another model on its Models page.");
group("proxy-key-mismatch", "R1: a pasted management key is confirmed before init", "Type the same key in both fields, or leave them empty to have the bus generate the key.");
group("proxy-lock-busy proxy-lock-invalid", "R4: one proxy key write at a time", "Wait for the running proxy write to finish and retry. An invalid lock file needs the owner to inspect the proxy state directory.");
group("proxy-not-configured proxy-unit-usage proxy-unit-exists proxy-unit-dropin proxy-unit-copy proxy-unit-path proxy-unit-missing proxy-unit-unhealthy proxy-unit-failed", "R1/R4: proxy user unit is explicit creation", "Initialize the proxy first, then install the unit from the Proxy page or with mosaic proxy unit install. An existing unit or drop-in is never replaced; remove it by hand. An unhealthy unit: read its journal by unit name and restart it.");
// proxy-ride first slice (Jason 2026-11-21): an instance's worker runs use another instance's proxy.
// Each next step is the same line packages/model-proxy/src/settings.mjs RIDE_LINES prints.
group("proxy-ride-invalid", "R1/R4: a ride file names one other instance of the same OS user", "proxy-ride.json is not a valid ride file, or it names this instance. Run mosaic proxy ride --off, then mosaic proxy ride --instance NAME with the instance whose proxy this one should use.");
group("proxy-ride-owner-missing", "R1/R4: the ridden instance owns a valid model proxy", "The instance named in proxy-ride.json has no valid model proxy, or it rides another instance itself. Set up the proxy on that instance, or run mosaic proxy ride --off.");
group("proxy-ride-conflict", "R1/R4: an instance has its own proxy or rides one, never both", "This instance has its own proxy.json and a proxy-ride.json. Run mosaic proxy ride --off to keep its own proxy.");
group("proxy-client-invalid proxy-client-refused proxy-client-response", "R1: proxy management answers", "Check the proxy unit is running and its port answers on the Proxy page, then retry. Upstream bodies are never shown.");
// proxy-scope slice 4 (Jason 2026-11-19): the Triage key lives with a bus that rides the owner's
// proxy, and a Triage session on a key past its retireAt ends before the key leaves the proxy.
group("triage-no-riding-bus", "R1/R4: the Triage key is issued and rotated by a bus that rides the owner's proxy", "Triage needs one organization that uses the owner's models. Start an organization that rides the owner's model proxy, then repair Triage access from Settings, Triage.");
group("triage-key-rotated", "R1/R6: a session ends when its Triage key's grace ends", "The Triage key was rotated and the old key's grace has ended. Send the next message; Triage starts again on the current key.");
group(
"block-recipient block-reply-required block-sender message-recipient-refused message-sender-refused tester-reply-required tester-thread-human user-reply-required user-thread-human",
"Role messaging: permitted sender, recipient and reply context",
+90 -4
View File
@@ -27,6 +27,7 @@ import { readSessions, sessionsFile, stopSession } from '../../seat/src/session.
import { compactionCount, storeDir } from '../../seat/src/compactions.mjs';
import { publishRemote, workingRepo, WorkingError } from '../../seat/src/working.mjs';
import { createLoader, hasModule } from '../../plugins/src/loader.mjs';
import { readWatchView } from '../../plugins/src/harness-watch.mjs';
import { pluginSettings } from '../../organization/src/plugin-settings.mjs';
import { loadManagementClient } from '../../model-proxy/src/client.mjs';
import { ProxyPage } from './proxy-page.mjs';
@@ -34,6 +35,7 @@ import { createModelCatalogue } from './model-catalogue.mjs';
import { sha256 } from '../../model-proxy/src/common.mjs';
import { initializeProxy, loadProxy } from '../../model-proxy/src/settings.mjs';
import { createProxyLock } from './proxy-lock.mjs';
import { ensureKey, readKey as readTriageKey, revokeKeys, sweepDue, sweepKeys } from './triage-key.mjs';
import { verifyRestart } from '../../model-proxy/src/admin.mjs';
import { ModelKeys, modelOrganization, stopModelRuns, MODEL_STOP_TIMEOUT_MS } from './model-keys.mjs';
import { instanceSettings } from '../../organization/src/instance-settings.mjs';
@@ -622,6 +624,10 @@ export async function startBroker({
// P1 validated sidecar from the trusted host; never a socket request setting.
proxy = null,
org = null, instanceDataRoot = null, instanceConfigDir = null,
// proxy-scope slice 4: the Triage key sweep on a riding org bus. intervalMs (default 30 s) and
// marginMs (default 30 s past a retiring key's retireAt, so the Console stops a session on it
// first) are in-process seams, as are now and hook (triage-key.mjs's clock and stage hook).
triageKeys = {},
// Host-only preparation/admission capabilities. The host builds its own from the organization writer and `trackers`
// (request-host.mjs); anything given here replaces the host's, and a capability nothing supplies leaves a setup wait.
requestServices = {},
@@ -657,6 +663,8 @@ export async function startBroker({
throw new BusError('invalid-host-request');
let modelProxyClient = null, proxyPage = null, proxyUsage = null, usageTimer = null, proxyLock = null, proxyPort = null;
let modelKeys = null, modelTimer = null, modelSweep = null, modelCatalogues = null;
// proxy-scope slice 4: the settings of the proxy this bus rides (boot, picked up or initialized).
let proxySettings = null, triageTimer = null, triageSweepRun = null;
const modelRuns = new Map();
let store, credentials, server, adapter, processing, syncTimer = null, syncPass = null, accessTimer = null, requestTimer = null, copyLaunch = null, copyRevoker = null, copyTimer = null;
// #1873 CB4: a revoke's stop waits for the broker; a start that fails before it exists stops nothing.
@@ -707,7 +715,7 @@ export async function startBroker({
proxyLock = createProxyLock({ stateDir: join(instanceDataRoot, 'proxy'), identity: `${instanceConfigDir}:${proxy.port}` });
}
modelProxyClient = await loadManagementClient({ configDir: org ? instanceConfigDir : organization.dir, proxy, receipt: proxyReceipt });
proxyPort = proxy.port;
proxyPort = proxy.port; proxySettings = proxy;
}
// Row 234 (#1740, r6 X4 third bullet): each host grant record gives its agent a store reference to
@@ -845,7 +853,7 @@ export async function startBroker({
// is no client: static text and a refusal code, never a path or a key.
let lateProxy = false, proxyCause = null, proxySync = Promise.resolve();
const dropLateProxy = cause => {
if (lateProxy) { modelProxyClient = null; proxyPage = null; proxyPort = null; proxyLock = null; lateProxy = false; modelCatalogues?.invalidate?.(); }
if (lateProxy) { modelProxyClient = null; proxyPage = null; proxyPort = null; proxyLock = null; proxySettings = null; lateProxy = false; modelCatalogues?.invalidate?.(); }
proxyCause = cause;
};
// The host refuses fixture-proxy-configured at boot when a dev fixture is recorded for the
@@ -879,11 +887,65 @@ export async function startBroker({
catch { return dropLateProxy('The shared proxy lock does not load (proxy-lock-invalid). Check the proxy state for this instance, then restart this bus.'); }
}
proxyCause = null;
proxySettings = found;
// Same port, key and lock still load: keep the client and its page, which holds login states.
if (modelProxyClient) return;
modelProxyClient = client; proxyPort = found.port; proxyLock = lock; lateProxy = true;
configureProxyPage(); modelCatalogues?.invalidate?.();
}
// proxy-scope slice 4: what triage-key.mjs needs from a riding org bus. A key error becomes a bus
// refusal the table already holds: proxy codes it knows pass through, the rest name the key
// store's code in the reason only. Neither carries a key, a path or upstream text.
// The verb checks the proxy first, as proxy.fence.clear does; the sweep uses what this bus holds
// and never picks a proxy up on its own timer.
const triageOptions = async ({ sync = true } = {}) => {
if (sync) await syncProxy();
if (org === null || !organization || !proxyLock || !modelProxyClient || !proxySettings || typeof instanceConfigDir !== 'string' || typeof instanceDataRoot !== 'string')
throw new BusError('triage-no-riding-bus');
return { configDir: instanceConfigDir, system: { dataRoot: instanceDataRoot, proxy: proxySettings }, client: modelProxyClient, lock: proxyLock,
...(triageKeys.now ? { now: triageKeys.now } : {}), ...(triageKeys.hook ? { hook: triageKeys.hook } : {}) };
};
const TRIAGE_PASS = new Set(['proxy-lock-busy', 'proxy-lock-invalid', 'proxy-lock-uncertain', 'proxy-client-invalid', 'proxy-client-refused', 'proxy-client-response', 'proxy-management-key-invalid']);
const TRIAGE_MAP = { 'triage-key-no-proxy': 'triage-no-riding-bus', 'triage-key-raced': 'proxy-lock-busy', 'triage-key-unconfirmed': 'proxy-lock-uncertain',
'proxy-uncertain': 'proxy-lock-uncertain', 'proxy-client-down': 'proxy-client-response', 'proxy-client-redirect': 'proxy-client-response', 'triage-key-proxy-open': 'proxy-not-configured' };
const triageRefusal = error => {
if (error instanceof BusError) return error;
const code = typeof error?.code === 'string' ? error.code : '';
if (TRIAGE_PASS.has(code)) return new BusError(code);
if (Object.hasOwn(TRIAGE_MAP, code)) return new BusError(TRIAGE_MAP[code], code);
return new BusError('storage-unavailable', /^triage-key-[a-z-]+$/.test(code) ? code : 'triage-key-failed');
};
// The sweep: a cheap read of the file first, so the proxy is asked only when something is due;
// sweepKeys decides again under the lock. It never issues: no file, nothing.
const triageMarginMs = Number.isSafeInteger(triageKeys.marginMs) && triageKeys.marginMs >= 0 ? triageKeys.marginMs : 30_000;
// A refusal that repeats on every tick is logged when it first appears or changes, and then once
// an hour while it stays the same; any other outcome clears it.
let triageLastRefusal = null;
const sweepRefused = (now, entry) => {
const sig = `${entry.code}|${entry.reason ?? ''}`;
if (triageLastRefusal?.sig === sig && now - triageLastRefusal.at < 3_600_000) return;
triageLastRefusal = { sig, at: now };
proxyReceipt({ action: 'triage.key.sweep-refused', ...entry });
};
const sweepTriage = async () => {
let opts;
try { opts = await triageOptions({ sync: false }); } catch { return { action: 'none' }; }
const now = (opts.now ?? Date.now)();
let record;
try { record = readTriageKey(opts.configDir); } catch { sweepRefused(now, { code: 'triage-key-unreadable' }); return { action: 'none' }; }
if (!sweepDue(record, now, triageMarginMs)) { triageLastRefusal = null; return { action: 'none' }; }
try {
const result = await sweepKeys(opts, { marginMs: triageMarginMs });
triageLastRefusal = null;
if (result.action !== 'none') proxyReceipt({ action: `triage.key.${result.action}`, keyId: result.keyId ?? null, addedKeyIds: result.addedKeyIds, removedKeyIds: result.removedKeyIds });
return result;
} catch (error) {
const refusal = triageRefusal(error);
sweepRefused(now, { code: refusal.code, ...(typeof refusal.reason === 'string' ? { reason: refusal.reason } : {}) });
return { action: 'refused', code: refusal.code };
}
};
const triageSweepOnce = () => (triageSweepRun ??= sweepTriage().finally(() => { triageSweepRun = null; }));
// One check at a time; an unexpected failure drops a late client, never keeps it.
const syncProxy = () => (proxySync = proxySync.then(pickProxy).catch(() => dropLateProxy('The model proxy could not be checked (proxy-down). Try again shortly.')));
// Unavailable with no client: the catalogue names the cause instead of a bare unavailable.
@@ -1332,7 +1394,9 @@ export async function startBroker({
throw new TemplateError('invalid-request', 'template.save takes preview: true or the preview digest');
}
if(verb==='limits')return withTokens(id,settings.read(id));
if(verb==='plugins')return {...plugins.read(id), available:{harness:levels.names ?? []}};
// #1873 W3: harnessWatch is what the host's harness watch last wrote under the org's data root,
// read only; a missing or bad file gives each harness "not checked yet" (readWatchView).
if(verb==='plugins')return {...plugins.read(id), available:{harness:levels.names ?? []}, harnessWatch:readWatchView({dataRoot, names:levels.names ?? [], paths:[instanceRoot]})};
// Row 256 (#1761): the actor is the reader, so endProtected is the delete check for that reader.
if(verb==='instances')return instances.read(id,actor);
// Row 65 (mos-0840): the Console's routing table.
@@ -1419,7 +1483,7 @@ export async function startBroker({
const lock = org && !proxyLock ? createProxyLock({ stateDir: join(instanceDataRoot, 'proxy'), identity: `${instanceConfigDir}:${settings.port}` }) : proxyLock;
// proxy-pickup: a client this bus did not boot with is checked again like a picked-up one.
if (!proxy) { lateProxy = true; proxyCause = null; }
modelProxyClient = client; proxyPort = settings.port; proxyLock = lock;
modelProxyClient = client; proxyPort = settings.port; proxyLock = lock; proxySettings = settings;
configureProxyPage(); modelCatalogues?.invalidate?.();
return settings;
} : null,
@@ -1430,6 +1494,15 @@ export async function startBroker({
if (!proxyLock || !modelProxyClient) throw new BusError('not-configured', proxyCause ?? undefined);
return proxyLock.clear({ restarted: true, read: () => modelProxyClient.getKeys(), verify: previous => verifyRestart(evidence, { fence: previous, port: proxyPort }) });
} : null,
// proxy-scope slice 4 (Jason 2026-11-19): ensure or revoke the instance's Triage key through the
// proxy this org bus rides, under the bus's own client and instance proxy lock. The answer is
// ids only. A no-org bus, or an org bus with no proxy client and lock, refuses
// triage-no-riding-bus.
proxyTriageKey: async (op) => {
const opts = await triageOptions();
try { return op === 'ensure' ? await ensureKey(opts) : await revokeKeys(opts); }
catch (error) { throw triageRefusal(error); }
},
credentialVerbs: organization
? credentialVerbs({ credentials, writer: organizationWriter({ dir: organization.dir, rolesDir: organization.rolesDir, dataRoot:instanceRoot ?? dataRoot }),
organizations: () => Object.keys(businesses), boot: copies?.definitions ? { org: copies.org, definitions: copies.definitions } : null })
@@ -1737,6 +1810,12 @@ export async function startBroker({
}, access.intervalMs ?? ACCESS_SWEEP_MS);
accessTimer.unref();
}
// proxy-scope slice 4: only an org bus sweeps the Triage key; each tick checks it still rides.
if (org !== null && organization) {
triageTimer = setInterval(() => { triageSweepOnce(); },
Number.isSafeInteger(triageKeys.intervalMs) && triageKeys.intervalMs > 0 ? triageKeys.intervalMs : 30_000);
triageTimer.unref();
}
if (modelKeys) {
modelTimer = setInterval(() => {
if (modelSweep) return;
@@ -1864,6 +1943,8 @@ export async function startBroker({
},
threadsState: (business) => broker.threadsState(business),
threadIdle: (r) => broker.threadIdle(r),
// #1873 chat-fail F2: the plain line a refused thread start or a proxy-stall stop leaves.
failureNotice: (r) => broker.failureNotice(r),
// Row 125 (#1619, REQ-LIMIT-4): the launcher asks the limit checks before every launch.
mayLaunch: async ({ business, instance, provider, model, account = null }) => {
processingWork();
@@ -1903,6 +1984,8 @@ export async function startBroker({
definitionScan,
launches: bound,
readers: readCaps,
// proxy-scope slice 4: one Triage key sweep now, as the timer runs it (tests).
triageKeySweep: () => triageSweepOnce(),
async close() {
clearInterval(copyTimer);
brokerUp(null);
@@ -1911,6 +1994,8 @@ export async function startBroker({
await proxyUsage.close();
clearInterval(modelTimer);
await modelSweep;
clearInterval(triageTimer);
await triageSweepRun;
clearInterval(accessTimer);
await stopSync();
await server.close();
@@ -1928,6 +2013,7 @@ export async function startBroker({
clearInterval(usageTimer);
await proxyUsage?.close();
clearInterval(modelTimer);
clearInterval(triageTimer);
clearInterval(accessTimer);
await server?.close();
await adapter?.close?.();
+2
View File
@@ -155,6 +155,8 @@ export async function serve({ broker, path, authenticateHuman = null, authentica
// Row 236 (#1742): the grants verbs may call Gitea several times, 15 s each at most.
GRANTS_VERBS.includes(r.verb) ||
r.verb === 'proxy.init' ||
// proxy-scope slice 4: the Triage key verb waits on the proxy lock and the proxy.
r.verb === 'proxy.triage.key' ||
(r.verb === 'decision.raise' && (Object.hasOwn(r.args ?? {}, 'spec_path') || Object.hasOwn(r.args ?? {}, 'spec_commit') ||
Object.hasOwn(r.args ?? {}, 'grant')))) {
const own = transient;
@@ -7,6 +7,12 @@
// <configDir>/secrets/triage/model (directory 0700, file 0600, the running user's)
// { version, key, keyId, issuedAt, rotateBy, retiring: [{ key, keyId, issuedAt, retireAt }] }
//
// worker-key-purpose (S3 of reports/worker-proxy-simple.md, Jason's ruling 2026-11-21): the same
// file shape and checks serve the worker key. A purpose, `triage` (the default, unchanged) or
// `worker`, picks the key prefix, the directory, the wording and the code prefix:
// triage mst-<tag>-triage-<43> <configDir>/secrets/triage/model triage-key-*
// worker msw-<tag>-worker-<43> <configDir>/secrets/worker/model worker-key-*
//
// Messages name files, never values.
import { closeSync, constants, fchmodSync, fstatSync, lstatSync, mkdirSync, openSync, readFileSync } from "node:fs";
import { createHash } from "node:crypto";
@@ -29,16 +35,30 @@ export class TriageKeyError extends Error {
this.exitCode = exitCode;
}
}
export const unreadable = (why) => new TriageKeyError("triage-key-unreadable", `the Triage key store is unreadable: ${why}`);
// What a purpose changes. `code` prefixes every refusal code this module and triage-key.mjs raise
// for that purpose; `head` and `name` make the key prefix; `dir` is the directory under secrets.
export const PURPOSES = Object.freeze({
triage: Object.freeze({ name: "triage", head: "mst", dir: "triage", code: "triage-key", store: "the Triage key store", dirWords: "the triage secrets directory", label: "Triage" }),
worker: Object.freeze({ name: "worker", head: "msw", dir: "worker", code: "worker-key", store: "the worker key store", dirWords: "the worker secrets directory", label: "worker" }),
});
export function purposeOf(purpose = "triage") {
if (!Object.hasOwn(PURPOSES, purpose)) throw new TypeError("the key purpose must be triage or worker");
return PURPOSES[purpose];
}
// A refusal for `purpose`: `suffix` is the part after the prefix, so ("worker", "raced") is
// worker-key-raced. The Triage codes are exactly the codes this module raised before purposes.
export const keyError = (purpose, suffix, message, exitCode) => new TriageKeyError(`${purposeOf(purpose).code}-${suffix}`, message, exitCode);
export const unreadable = (why, purpose = "triage") => keyError(purpose, "unreadable", `${purposeOf(purpose).store} is unreadable: ${why}`);
export const keyIdOf = (key) => createHash("sha256").update(key).digest("hex").slice(0, 12);
export const instanceTag = (configDir) => createHash("sha256").update(configDir).digest("hex").slice(0, 12);
export const keyPrefix = (configDir) => `mst-${instanceTag(configDir)}-triage-`;
const ownKey = (configDir) => (k) => typeof k === "string" && k.startsWith(keyPrefix(configDir)) && SECRET.test(k.slice(keyPrefix(configDir).length));
export const keyPrefix = (configDir, purpose = "triage") => { const p = purposeOf(purpose); return `${p.head}-${instanceTag(configDir)}-${p.name}-`; };
const ownKey = (configDir, purpose) => (k) => typeof k === "string" && k.startsWith(keyPrefix(configDir, purpose)) && SECRET.test(k.slice(keyPrefix(configDir, purpose).length));
export function keyPaths(configDir) {
export function keyPaths(configDir, purpose = "triage") {
const secrets = join(configDir, "secrets");
const dir = join(secrets, "triage");
const dir = join(secrets, purposeOf(purpose).dir);
return { secrets, dir, file: join(dir, "model") };
}
@@ -47,8 +67,8 @@ const exactKeys = (v, keys) => plain(v) && Object.keys(v).length === keys.length
const canonicalTime = (s) => typeof s === "string" && ISO.test(s) && !Number.isNaN(Date.parse(s)) && new Date(s).toISOString() === s;
const canonicalDay = (s) => typeof s === "string" && DAY.test(s) && !Number.isNaN(Date.parse(`${s}T00:00:00Z`)) && new Date(`${s}T00:00:00Z`).toISOString().slice(0, 10) === s;
export function validRecord(doc, configDir) {
const own = ownKey(configDir);
export function validRecord(doc, configDir, purpose = "triage") {
const own = ownKey(configDir, purpose);
return exactKeys(doc, FIELDS)
&& doc.version === KEY_VERSION
&& own(doc.key) && doc.keyId === keyIdOf(doc.key)
@@ -62,23 +82,24 @@ export function validRecord(doc, configDir) {
const uidOf = (opts) => opts.uid ?? process.getuid();
// A real directory, the running user's, mode 0700. Missing: false, or made when `create` is on.
export function checkDir(dir, uid, what, { create }) {
export function checkDir(dir, uid, what, { create, purpose = "triage" }) {
const bad = (why) => unreadable(why, purpose);
let st;
try { st = lstatSync(dir); }
catch (e) {
if (e.code !== "ENOENT") throw unreadable(`${what} cannot be read`);
if (e.code !== "ENOENT") throw bad(`${what} cannot be read`);
if (!create) return false;
try { mkdirSync(dir, { mode: 0o700 }); } catch { throw unreadable(`${what} cannot be made`); }
try { mkdirSync(dir, { mode: 0o700 }); } catch { throw bad(`${what} cannot be made`); }
try {
const fd = openSync(dir, constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW);
try { fchmodSync(fd, 0o700); } finally { closeSync(fd); }
} catch { throw unreadable(`${what} cannot be made`); }
} catch { throw bad(`${what} cannot be made`); }
st = lstatSync(dir);
}
if (st.isSymbolicLink()) throw unreadable(`${what} is a link`);
if (!st.isDirectory()) throw unreadable(`${what} is not a directory`);
if (st.uid !== uid) throw unreadable(`${what} is not owned by this user`);
if ((st.mode & 0o777) !== 0o700) throw unreadable(`${what} mode is not 0700`);
if (st.isSymbolicLink()) throw bad(`${what} is a link`);
if (!st.isDirectory()) throw bad(`${what} is not a directory`);
if (st.uid !== uid) throw bad(`${what} is not owned by this user`);
if ((st.mode & 0o777) !== 0o700) throw bad(`${what} mode is not 0700`);
return true;
}
@@ -91,27 +112,30 @@ export function checkFile(fd, uid, what, fail = unreadable) {
return st;
}
// The stored record, or null when no key is issued. Throws triage-key-unreadable on anything unsafe.
// opts.uid (both directories and the file) and opts.fileUid (the file only) are test seams for the
// owner checks.
// The stored record, or null when no key is issued. Throws <purpose>-key-unreadable on anything
// unsafe. opts.purpose picks the key (default triage). opts.uid (both directories and the file) and
// opts.fileUid (the file only) are test seams for the owner checks.
export function readKey(configDir, opts = {}) {
const { secrets, dir, file } = keyPaths(configDir);
const purpose = opts.purpose ?? "triage";
const p = purposeOf(purpose);
const bad = (why) => unreadable(why, purpose);
const { secrets, dir, file } = keyPaths(configDir, purpose);
const uid = uidOf(opts);
if (!checkDir(secrets, uid, "the secrets directory", { create: false })) return null;
if (!checkDir(dir, uid, "the triage secrets directory", { create: false })) return null;
if (!checkDir(secrets, uid, "the secrets directory", { create: false, purpose })) return null;
if (!checkDir(dir, uid, p.dirWords, { create: false, purpose })) return null;
let fd;
try { fd = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK); }
catch (e) {
if (e.code === "ENOENT") return null;
if (e.code === "ELOOP") throw unreadable("the key file is a link");
throw unreadable("the key file cannot be opened");
if (e.code === "ELOOP") throw bad("the key file is a link");
throw bad("the key file cannot be opened");
}
try {
const st = checkFile(fd, opts.fileUid ?? uid, "the key file");
if (st.size > MAX_BYTES) throw unreadable("the key file is too large");
const st = checkFile(fd, opts.fileUid ?? uid, "the key file", bad);
if (st.size > MAX_BYTES) throw bad("the key file is too large");
let doc;
try { doc = JSON.parse(readFileSync(fd, "utf8")); } catch { throw unreadable("the key file is not JSON"); }
if (!validRecord(doc, configDir)) throw unreadable("the key file does not match the key shape");
try { doc = JSON.parse(readFileSync(fd, "utf8")); } catch { throw bad("the key file is not JSON"); }
if (!validRecord(doc, configDir, purpose)) throw bad("the key file does not match the key shape");
return doc;
} finally { closeSync(fd); }
}
+610
View File
@@ -0,0 +1,610 @@
// #1873 triage-key: Triage's own model proxy client key, issued from the owner's terminal
// (design report reports/triage-route-design.md slice 3, Jason's ruling 2026-11-18), and since
// proxy-scope slice 4 (Jason's ruling 2026-11-19) by the bus too: the verb proxy.triage.key
// (ensure, revoke) and a riding org bus's sweep call this module with the bus's own management
// client and instance proxy lock. It moved here from packages/cli/src so the bus can import it; the
// terminal verbs (packages/cli/src/triage.mjs) call it unchanged.
//
// <configDir>/secrets/triage/model (directory 0700, file 0600, the running user's)
// { version, key, keyId, issuedAt, rotateBy, retiring: [{ key, keyId, issuedAt, retireAt }] }
//
// The key is `mst-<instance tag>-triage-<43 base64url>`, where the tag is the first 12 hex of
// sha256(configDir), the tag the instance's own `msk-` sentinel carries. An org bus reconciles only
// its own `mso-<tag>-<org>-key-` names and a no-org bus only `msk-<tag>-`, so neither touches it.
//
// Every proxy write runs under the instance proxy lock (packages/bus/src/proxy-lock.mjs), the lock
// org buses take: read the list, check this instance's sentinel, fence, PUT, read back, confirm.
// The proxy changes first and the file second, as the bus does for an agent key. A crash between
// leaves a key in the Triage namespace that the file does not hold; the next issue, rotate or retire
// removes it, and status names it. A crash after the PUT and before the read-back leaves the lock's
// uncertain fence, which every proxy writer on the instance refuses until a human clears it
// (`mosaic proxy fence clear --proxy-restarted`).
//
// rotate records when the old key's grace ends; the key stays in the proxy until a later retire,
// rotate, issue or sweep removes it, and status says when that is due. A riding org bus runs
// sweepKeys on a timer: under the lock it reads the file again and rotates when rotateBy is due, or
// retires keys whose grace (plus a margin, so the Console stops a session on the key first) has
// passed. The sweep only replaces a key that exists; it never issues the first one.
//
// Nothing here prints, logs or records a key. Receipts and output carry key ids only (the first 12
// hex of sha256(key), the id the bus uses). Messages name files and options, never values.
// The Console reads the key at each Triage spawn through triage-key-file.mjs, never this module.
//
// worker-key-purpose (S3 of reports/worker-proxy-simple.md, Jason's ruling 2026-11-21): every export
// takes opts.purpose, `triage` (the default; nothing above changes) or `worker`. The worker key is
// `msw-<tag>-worker-<43 base64url>` in <configDir>/secrets/worker/model, and its refusal codes are
// the Triage codes with `worker-key-` for `triage-key-`. Each purpose removes names in its own
// namespace only; change() refuses any plan that would remove another name. Two worker-only rules:
// - ensureReady, the launch path: rotates when rotateBy has come and retires keys whose grace
// has passed, and
// - a retiring worker key whose grace has passed still stays while retireHeld says a live
// runtime started before the rotation (opts.runtimes, the processing-control list).
// ensureReady (both purposes) skips the lock when the plan it would run under the lock changes
// nothing, then probes the key. Nothing calls it for Triage yet.
import { closeSync, constants, fchmodSync, fsyncSync, lstatSync, openSync, readFileSync, readdirSync, renameSync, unlinkSync, writeSync } from "node:fs";
import { randomBytes } from "node:crypto";
import { join } from "node:path";
import { performance } from "node:perf_hooks";
import { isDeepStrictEqual } from "node:util";
import { createProxyLock } from "./proxy-lock.mjs";
import { loadManagementClient } from "../../model-proxy/src/client.mjs";
import { processIdentity, runtimeGroupState } from "../../processing-control/src/runtimes.mjs";
// #1873 triage-wiring: the key file's reader lives in triage-key-file.mjs (node built-ins only), so
// the Console reads the key without this module's proxy client. Re-exported here unchanged.
import { KEY_VERSION, MAX_RETIRING, PURPOSES, SECRET, TriageKeyError, checkDir, checkFile, instanceTag, keyError, keyIdOf, keyPaths, keyPrefix, purposeOf, readKey, unreadable, validRecord } from "./triage-key-file.mjs";
export { KEY_VERSION, PURPOSES, TriageKeyError, instanceTag, keyIdOf, keyPaths, keyPrefix, readKey, validRecord };
// The words that name a purpose's own verbs. Triage's are the words this module used before
// purposes; the worker key has no verbs, so its words name the launch.
const WORDS = {
triage: {
unrecorded: "the change was made but its receipt line was not written; run mosaic triage key status",
issued: "a Triage key is already issued; use mosaic triage key rotate",
notIssued: "no Triage key is issued; use mosaic triage key issue",
tooMany: "too many keys are retiring; run mosaic triage key retire once their grace has passed",
unwritten: (file, why, op) => `the proxy list changed but ${file} was not written (${why}); fix the file's directory, then run mosaic triage key ${op} again, which clears what this run left`,
unsynced: (file) => `${file} was written but its directory could not be synced, so it may not survive a power loss; run mosaic triage key status`,
raced: "the proxy's key list changed while this verb ran; run it again",
},
worker: {
unrecorded: "the change was made but its receipt line was not written; see <dataRoot>/proxy/mosaic-keys.receipts.jsonl",
issued: "a worker key is already issued",
notIssued: "no worker key is issued",
tooMany: "too many worker keys are retiring; they leave once their grace has passed and no older run is live",
unwritten: (file, why) => `the proxy list changed but ${file} was not written (${why}); fix the file's directory, then run the launch again, which clears what this run left`,
unsynced: (file) => `${file} was written but its directory could not be synced, so it may not survive a power loss; run the launch again`,
raced: "the proxy's key list changed while the worker key was being changed; run the launch again",
},
};
// The org key lifetime and the rotate grace default. The bus holds both as literals
// (packages/bus/src/model-keys.mjs #issue: 30 days to rotateBy; packages/bus/src/runtime.mjs:
// `proxy.rotateGraceMinutes ?? 120`) and exports neither, and this lane may not change the bus.
// packages/cli/tests/triage-key.test.mjs drives the bus's own ModelKeys with the same clock and
// proves these give the same rotateBy and retireAt, so a change on either side goes red there.
// The grace itself is read from proxy.json first, where the bus reads it.
export const KEY_LIFETIME_DAYS = 30;
export const DEFAULT_ROTATE_GRACE_MINUTES = 120;
const sentinelPrefix = (configDir) => `msk-${instanceTag(configDir)}-sentinel-`;
// Any name under the prefix is the purpose's to remove, well formed or not, as the bus treats its
// own `msk-<tag>-` names. Only a well-formed key may be stored in the file.
const inNamespace = (configDir, purpose = "triage") => (k) => typeof k === "string" && k.startsWith(keyPrefix(configDir, purpose));
// Both directories, checked (and the purpose's one made) before any proxy write, so a store that
// cannot be written refuses while the proxy is still untouched.
function prepareDirs(configDir, uid, purpose = "triage") {
const { secrets, dir } = keyPaths(configDir, purpose);
if (!checkDir(secrets, uid, "the secrets directory", { create: false, purpose })) throw unreadable("the secrets directory is missing", purpose);
checkDir(dir, uid, purposeOf(purpose).dirWords, { create: true, purpose });
}
// A temp file writeKey left when it was killed before its rename. It may hold a key.
const TEMP = /^\.model\.\d+\.[0-9a-f]{12}\.tmp$/;
const tempsIn = (dir) => { try { return readdirSync(dir).filter((n) => TEMP.test(n)); } catch { return []; } };
// Under the lock no other writer of this purpose runs, so every temp file left in the directory is dead.
function removeTemps(configDir, purpose = "triage") {
const { dir } = keyPaths(configDir, purpose);
for (const name of tempsIn(dir)) {
try { if (!lstatSync(join(dir, name)).isDirectory()) unlinkSync(join(dir, name)); }
catch { throw unreadable(`a leftover temp file in ${purposeOf(purpose).dirWords} cannot be removed`, purpose); }
}
}
// A directory sync that failed after the rename: the file is in place but may not be durable.
class NotSynced extends Error {}
// Atomic: a 0600 temp file in the same directory, fsync, rename, fsync of the directory.
function writeKey(configDir, record, uid, hook = () => {}, purpose = "triage") {
const { dir, file } = keyPaths(configDir, purpose);
prepareDirs(configDir, uid, purpose);
const temp = join(dir, `.model.${process.pid}.${randomBytes(6).toString("hex")}.tmp`);
let fd;
try { fd = openSync(temp, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600); }
catch { throw unreadable("the key file cannot be written", purpose); }
try {
try { fchmodSync(fd, 0o600); writeSync(fd, `${JSON.stringify(record, null, 2)}\n`); fsyncSync(fd); }
finally { closeSync(fd); }
hook("rename");
renameSync(temp, file);
} catch { try { unlinkSync(temp); } catch {} throw unreadable("the key file cannot be written", purpose); }
try {
hook("sync");
const dfd = openSync(dir, constants.O_RDONLY | constants.O_DIRECTORY);
try { fsyncSync(dfd); } finally { closeSync(dfd); }
} catch { throw new NotSynced(); }
}
// One line per write in the proxy lock's receipts file, ids only. `receiptsReady` runs under the
// lock before any change, so an unsafe receipts file refuses while nothing has changed yet.
const RECEIPTS_FLAGS = constants.O_WRONLY | constants.O_APPEND | constants.O_CREAT | constants.O_NOFOLLOW | constants.O_NONBLOCK;
const receiptsUnsafe = (purpose) => (why) => keyError(purpose, "receipts-unsafe", `${why}; nothing was changed`);
function receiptsReady(stateDir, uid, purpose = "triage") {
let fd;
try { fd = openSync(join(stateDir, "mosaic-keys.receipts.jsonl"), RECEIPTS_FLAGS, 0o600); }
catch { throw receiptsUnsafe(purpose)("the proxy receipts file cannot be opened"); }
try { checkFile(fd, uid, "the proxy receipts file", receiptsUnsafe(purpose)); } finally { closeSync(fd); }
}
function receipt(stateDir, uid, entry, purpose = "triage") {
const unrecorded = keyError(purpose, "unrecorded", WORDS[purpose].unrecorded);
let fd;
try { fd = openSync(join(stateDir, "mosaic-keys.receipts.jsonl"), RECEIPTS_FLAGS, 0o600); } catch { throw unrecorded; }
try {
checkFile(fd, uid, "the proxy receipts file", () => unrecorded);
writeSync(fd, `${JSON.stringify(entry)}\n`);
fsyncSync(fd);
} catch { throw unrecorded; }
finally { closeSync(fd); }
}
// The grace the bus uses: proxy.json's rotateGraceMinutes, or the bus default.
export const graceMinutes = (proxy) => proxy?.rotateGraceMinutes ?? DEFAULT_ROTATE_GRACE_MINUTES;
export const rotateByFor = (nowMs) => new Date(nowMs + KEY_LIFETIME_DAYS * 86400_000).toISOString().slice(0, 10);
export const retireAtFor = (nowMs, proxy) => new Date(nowMs + graceMinutes(proxy) * 60_000).toISOString();
const proxyFailure = (code, purpose = "triage") => {
const words = {
"proxy-lock-busy": "another proxy writer holds the instance proxy lock; try again when it ends, and if no writer is running, see mosaic proxy fence status",
"proxy-lock-uncertain": "the instance proxy lock is in a state this verb will not guess about; check <dataRoot>/proxy by hand",
"proxy-lock-invalid": "the instance proxy lock directory is missing or unsafe, or names another proxy",
"proxy-uncertain": "an earlier proxy write has an uncertain outcome; see mosaic proxy fence status",
"proxy-management-key-invalid": "the proxy management key file is missing or unsafe",
"proxy-client-invalid": "the proxy settings are not usable",
"proxy-client-down": "the proxy did not answer",
"proxy-client-refused": "the proxy refused the management request",
"proxy-client-redirect": "the proxy answered with a redirect",
"proxy-client-response": "the proxy gave an answer this verb cannot use",
};
return new TriageKeyError(code === "triage-key-failed" ? `${purposeOf(purpose).code}-failed` : code, words[code] ?? "the proxy write failed");
};
// ensureReady's probe refusals, the codes a bus launch uses for the same answers.
const probeFailure = (code) => new TriageKeyError(code, code === "proxy-down" ? "the proxy did not answer the key probe" : "the proxy did not accept the key");
// opts.uid, opts.now and opts.hook(stage) are test seams; a stage hook may throw or kill the process.
// opts.client and opts.lock: the bus passes its own management client and instance proxy lock, so a
// bus change runs under the lock object its model key sweep uses and reads no key file of its own.
// opts.purpose: "triage" (default) or "worker". opts.runtimes (worker only): () => the
// processing-control runtime list, listRuntimes(dataRoot); without it no worker key whose grace has
// passed is removed. opts.retireGuard: test seams for retireHeld ({ marginMs, live, startedAt }).
// proxy-ride first slice (Jason 2026-11-21): a riding instance's system.proxy carries `owner`
// ({ instance, configDir, dataRoot }) from scripts/mosaic-config.mjs. Without it, the instance's own.
const proxyConfigDir = (configDir, system) => system?.proxy?.owner?.configDir ?? configDir;
const proxyStateDir = (system) => join(system?.proxy?.owner?.dataRoot ?? system.dataRoot, "proxy");
function context({ configDir, system, uid, fileUid, now, hook, client, lock, purpose = "triage", runtimes, retireGuard, planSeam }) {
const p = purposeOf(purpose);
if (!system?.proxy || !Number.isInteger(system.proxy.port)) throw keyError(purpose, "no-proxy", "the instance has no model proxy set up (proxy.json)");
return {
configDir, dataRoot: system.dataRoot, proxy: system.proxy, stateDir: proxyStateDir(system),
// proxy-ride: the instance whose proxy this is. Its lock identity, sentinel and management key
// are the owner's; the key file, key names and namespace stay this instance's (configDir).
proxyConfigDir: proxyConfigDir(configDir, system),
uid: uid ?? process.getuid(), fileUid, now: now ?? Date.now, hook: hook ?? (() => {}), client, lock,
purpose, p, words: WORDS[purpose],
// Tests only: rewrites a plan's result, to prove change()'s namespace check refuses it.
planSeam: typeof planSeam === "function" ? planSeam : (result) => result,
// A Triage key leaves when its grace ends. A worker key also waits for older runs to end.
held: purpose === "worker" ? (record) => retireHeld({ runtimes, rotatedAtMs: Date.parse(record.issuedAt), ...(retireGuard ?? {}) }) : () => false,
};
}
function lockFor(c) {
if (c.lock) return c.lock;
try { return createProxyLock({ stateDir: c.stateDir, identity: `${c.proxyConfigDir}:${c.proxy.port}` }); }
catch (e) { throw proxyFailure(e.code ?? "proxy-lock-invalid", c.purpose); }
}
async function clientFor(c) {
if (c.client) return c.client;
try { return await loadManagementClient({ configDir: c.proxyConfigDir, proxy: c.proxy }); }
catch (e) { throw proxyFailure(e.code ?? "proxy-management-key-invalid", c.purpose); }
}
function checkSentinel(c, keys) {
const prefix = sentinelPrefix(c.proxyConfigDir);
if (!keys.some((k) => k.startsWith(prefix) && SECRET.test(k.slice(prefix.length)))) {
throw keyError(c.purpose, "proxy-open", "the proxy's key list has no sentinel for this instance, so it may be another proxy");
}
}
// Under the instance proxy lock: read the list, check the sentinel, run `plan`, write the result.
// `plan(keys, record)` returns { after, record, remove, op, done }; `record` null leaves the file
// alone, `remove` true unlinks it (revoke), `op` names the change when the plan picks it (sweep).
// The proxy changes first, then the file, then one receipt line.
async function change(c, opName, plan) {
const lock = lockFor(c);
const client = await clientFor(c);
try {
return await lock.run(async () => {
const before = await client.getKeys();
checkSentinel(c, before);
const current = readKey(c.configDir, { uid: c.uid, fileUid: c.fileUid, purpose: c.purpose });
const result = c.planSeam(plan(before, current), before);
const op = result.op ?? opName;
const remove = result.remove === true && current !== null;
if (result.record) prepareDirs(c.configDir, c.uid, c.purpose);
removeTemps(c.configDir, c.purpose);
receiptsReady(c.stateDir, c.uid, c.purpose);
const after = result.after;
const { removed, added } = diff(before, after);
// A key in another namespace never leaves the list.
if (removed.some((k) => !inNamespace(c.configDir, c.purpose)(k))) throw keyError(c.purpose, "failed", `the change would remove a key outside the ${c.p.label} namespace`);
if (added.length || removed.length) {
// As the bus does: a list that moved since the first read is not overwritten.
if (!isDeepStrictEqual(await client.getKeys(), before)) throw keyError(c.purpose, "raced", c.words.raced);
lock.fence({ org: null, op: `${c.p.name}.${op}` });
await client.putKeys(after);
c.hook("put");
const confirmed = await client.getKeys();
if (!confirmed.some((k) => k.startsWith(sentinelPrefix(c.proxyConfigDir))) || added.some((k) => !confirmed.includes(k)) || removed.some((k) => confirmed.includes(k))) {
throw keyError(c.purpose, "unconfirmed", "the proxy did not confirm the change; the instance proxy fence stays set, see mosaic proxy fence status");
}
lock.confirm();
}
c.hook("write");
let unsynced = false;
if (result.record || remove) {
try { if (remove) unlinkKey(c.configDir, c.hook, c.purpose); else writeKey(c.configDir, result.record, c.uid, c.hook, c.purpose); }
catch (e) {
if (e instanceof NotSynced) unsynced = true;
else throw keyError(c.purpose, "unwritten", c.words.unwritten(keyPaths(c.configDir, c.purpose).file, e.message, op));
}
}
c.hook("receipt");
// A run that changed neither the proxy nor the file leaves no receipt.
if (added.length || removed.length || result.record || remove) receipt(c.stateDir, c.uid, { op: `${c.p.name}.key.${op}`, time: new Date(c.now()).toISOString(), addedKeyIds: added.map(keyIdOf), removedKeyIds: removed.map(keyIdOf) }, c.purpose);
// The file is in place and the receipt written; only its durability is in doubt.
if (unsynced) throw keyError(c.purpose, "unsynced", c.words.unsynced(keyPaths(c.configDir, c.purpose).file));
return { ...result.done, addedKeyIds: added.map(keyIdOf), removedKeyIds: removed.map(keyIdOf) };
});
} catch (e) {
if (e instanceof TriageKeyError) throw e;
throw proxyFailure(typeof e?.code === "string" && /^proxy-[a-z-]+$/.test(e.code) ? e.code : "triage-key-failed", c.purpose);
}
}
const diff = (before, after) => ({ removed: before.filter((k) => !after.includes(k)), added: after.filter((k) => !before.includes(k)) });
// True when change() would write the proxy list, the file or a receipt for this plan.
const writes = (before, current, result) => {
const { removed, added } = diff(before, result.after);
return added.length > 0 || removed.length > 0 || Boolean(result.record) || (result.remove === true && current !== null);
};
// Revoke's file step: unlink, then sync the directory, with writeKey's stages and failures.
function unlinkKey(configDir, hook = () => {}, purpose = "triage") {
const { dir, file } = keyPaths(configDir, purpose);
try { hook("rename"); unlinkSync(file); }
catch (e) { if (e?.code !== "ENOENT") throw unreadable("the key file cannot be removed", purpose); }
try {
hook("sync");
const dfd = openSync(dir, constants.O_RDONLY | constants.O_DIRECTORY);
try { fsyncSync(dfd); } finally { closeSync(dfd); }
} catch { throw new NotSynced(); }
}
const newKey = (configDir, purpose = "triage") => keyPrefix(configDir, purpose) + randomBytes(32).toString("base64url");
// Keys in the purpose's namespace that the file does not hold: left by a crash between the PUT and
// the file write, or by a file that was removed by hand.
const orphansOf = (configDir, keys, record, purpose = "triage") => {
const held = record ? [record.key, ...record.retiring.map((r) => r.key)] : [];
return keys.filter((k) => inNamespace(configDir, purpose)(k) && !held.includes(k));
};
export async function issueKey(opts) {
const c = context(opts);
return change(c, "issue", (before, current) => {
if (current) throw keyError(c.purpose, "issued", c.words.issued);
const orphans = orphansOf(c.configDir, before, null, c.purpose);
const key = newKey(c.configDir, c.purpose), nowMs = c.now();
const record = { version: KEY_VERSION, key, keyId: keyIdOf(key), issuedAt: new Date(nowMs).toISOString(), rotateBy: rotateByFor(nowMs), retiring: [] };
return { after: [...before.filter((k) => !orphans.includes(k)), key], record, done: { keyId: record.keyId, rotateBy: record.rotateBy } };
});
}
// Retiring keys whose grace (plus marginMs, 0 for the terminal verbs) has passed, or that the proxy
// no longer holds. For a worker key, a key whose grace has passed stays while c.held(record) says
// an older run may still use it; a key the proxy no longer holds goes from the file either way.
const graceOver = (record, nowMs, marginMs) => record.retiring.some((r) => Date.parse(r.retireAt) + marginMs <= nowMs);
const dueOf = (record, keys, nowMs, marginMs = 0, held = false) => record.retiring.filter((r) => (!held && Date.parse(r.retireAt) + marginMs <= nowMs) || !keys.includes(r.key));
const heldFor = (c, record, nowMs, marginMs) => graceOver(record, nowMs, marginMs) && c.held(record);
const rotateDueAt = (record) => Date.parse(`${record.rotateBy}T00:00:00Z`);
// The rotate plan, shared by the terminal verb and the sweep.
function rotatePlan(c, before, current, nowMs, marginMs = 0) {
const orphans = orphansOf(c.configDir, before, current, c.purpose);
const due = dueOf(current, before, nowMs, marginMs, heldFor(c, current, nowMs, marginMs));
const keep = before.includes(current.key);
const retiring = [...current.retiring.filter((r) => !due.includes(r)),
...(keep ? [{ key: current.key, keyId: current.keyId, issuedAt: current.issuedAt, retireAt: retireAtFor(nowMs, c.proxy) }] : [])];
if (retiring.length > MAX_RETIRING) throw keyError(c.purpose, "failed", c.words.tooMany);
const key = newKey(c.configDir, c.purpose);
const record = { version: KEY_VERSION, key, keyId: keyIdOf(key), issuedAt: new Date(nowMs).toISOString(), rotateBy: rotateByFor(nowMs), retiring };
const drop = [...orphans, ...due.map((r) => r.key)];
return {
after: [...before.filter((k) => !drop.includes(k)), key], record,
done: { keyId: record.keyId, rotateBy: record.rotateBy, old: keep ? { keyId: current.keyId, retireAt: retiring.at(-1).retireAt } : { keyId: current.keyId, lost: true } },
};
}
export async function rotateKey(opts) {
const c = context(opts);
return change(c, "rotate", (before, current) => {
if (!current) throw keyError(c.purpose, "not-issued", c.words.notIssued);
return rotatePlan(c, before, current, c.now());
});
}
function retirePlan(c, before, current, nowMs, marginMs = 0) {
const orphans = orphansOf(c.configDir, before, current, c.purpose);
const due = current ? dueOf(current, before, nowMs, marginMs, heldFor(c, current, nowMs, marginMs)) : [];
const record = current && due.length ? { ...current, retiring: current.retiring.filter((r) => !due.includes(r)) } : null;
const drop = [...orphans, ...due.map((r) => r.key)];
return { after: before.filter((k) => !drop.includes(k)), record, done: { droppedKeyIds: due.map((r) => r.keyId) } };
}
export async function retireKeys(opts) {
const c = context(opts);
return change(c, "retire", (before, current) => retirePlan(c, before, current, c.now()));
}
// proxy-scope slice 4: the bus verb proxy.triage.key op ensure. Leaves a held key alone, issues one
// when none exists, replaces one the proxy lost. Never prints the key; the answer is ids only.
// issued no file: a new key (keys in the namespace the file does not hold are removed)
// held the proxy holds the file's key: only orphans and retiring keys the proxy lost go
// replaced the proxy does not hold the file's key: a new key, the old one marked lost
function ensurePlan(c, before, current, nowMs) {
const orphans = orphansOf(c.configDir, before, current, c.purpose);
const fresh = (retiring) => {
const key = newKey(c.configDir, c.purpose);
return { version: KEY_VERSION, key, keyId: keyIdOf(key), issuedAt: new Date(nowMs).toISOString(), rotateBy: rotateByFor(nowMs), retiring };
};
if (!current) {
const record = fresh([]);
return { after: [...before.filter((k) => !orphans.includes(k)), record.key], record, done: { action: "issued", keyId: record.keyId, rotateBy: record.rotateBy } };
}
const lost = current.retiring.filter((r) => !before.includes(r.key));
const retiring = current.retiring.filter((r) => !lost.includes(r));
if (before.includes(current.key)) {
return {
after: before.filter((k) => !orphans.includes(k)), record: lost.length ? { ...current, retiring } : null,
done: { action: "held", keyId: current.keyId, rotateBy: current.rotateBy },
};
}
const record = fresh(retiring);
return {
after: [...before.filter((k) => !orphans.includes(k)), record.key], record,
done: { action: "replaced", keyId: record.keyId, rotateBy: record.rotateBy, old: { keyId: current.keyId, lost: true } },
};
}
export async function ensureKey(opts) {
const c = context(opts);
return change(c, "ensure", (before, current) => ensurePlan(c, before, current, c.now()));
}
// worker-key-purpose: the launch's plan. Triage: ensureKey's plan. Worker: the same, and on a held
// key it also rotates once rotateBy has come and retires keys whose grace has passed (unless an
// older run is live, see retireHeld). The worker key has no sweep; launches are its timer.
function readyPlan(c, before, current, nowMs) {
const base = ensurePlan(c, before, current, nowMs);
if (c.purpose !== "worker" || base.done.action !== "held") return base;
if (rotateDueAt(current) <= nowMs) {
// When older runs hold so many retiring keys that one more would pass MAX_RETIRING, the launch
// keeps the current key (still in the list) and rotates at a later launch.
let plan = null;
try { plan = rotatePlan(c, before, current, nowMs); }
catch (e) { if (!(e instanceof TriageKeyError && e.message === c.words.tooMany)) throw e; }
if (plan) return { ...plan, op: "rotate", done: { action: "rotated", ...plan.done } };
}
// Retire only for a key whose grace has passed and that the list still holds; orphans and lost
// keys are the ensure plan's.
const plan = retirePlan(c, before, current, nowMs);
if (!plan.record || !current.retiring.some((r) => before.includes(r.key) && !plan.record.retiring.includes(r))) return base;
return { ...plan, op: "retire", done: { action: "retired", keyId: current.keyId, rotateBy: current.rotateBy, ...plan.done } };
}
// model-keys.mjs's ISSUE_PROBE_TIMEOUT_MS, ISSUE_REPROBE_MIN_MS and REVOKE_POLL_MS, copied so the key
// verbs do not load the broker; worker-key.test.mjs pins them equal.
export const PROBE_TIMING = Object.freeze({ timeoutMs: 2000, reprobeMinMs: 250, pollMs: 25 });
const pause = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
// As ModelKeys.preflight probes a just-issued key: CLIProxyAPI applies a key-list PUT by an async
// reload after it answers, so only a 401 is probed again, until a short deadline.
async function probeReady(client, key) {
const deadline = performance.now() + PROBE_TIMING.timeoutMs;
let status, first = true;
for (;;) {
const options = first ? undefined : { timeout: Math.max(PROBE_TIMING.reprobeMinMs, Math.ceil(deadline - performance.now())) };
first = false;
try { status = await client.probeKey(key, options); }
catch { throw probeFailure("proxy-down"); }
if (status !== 401 || performance.now() >= deadline) break;
await pause(PROBE_TIMING.pollMs);
}
if (status < 200 || status >= 300) throw probeFailure("proxy-key-refused");
}
// worker-key-purpose: ensure a usable key for a launch, without the lock when nothing needs writing.
// Fast path: read the file and the list (no lock, no write, not even the lock's identity file), run
// the plan change() would run, and when it writes nothing, no temp file is left and no fence is set,
// probe the file's key. Otherwise change() takes the lock, reads both again, decides again and
// writes; then the key is probed. Two launches that both see "no key" serialize on the lock, and
// the second finds the first's key held. A fast-path probe the proxy refuses (a list writer that
// dropped the key after the read) goes to the lock once, which re-adds the key, and refuses only
// when the probe after that fails too. The answer is ids only; the caller reads the key with
// readKey and must call again when readKey's keyId is not the answer's keyId: a rotate by another
// launch in between leaves a newer key that this call did not probe. opts.hook also sees "fast"
// (no lock) and "locked" (the slow path) for tests.
export async function ensureReady(opts) {
const c = context(opts);
const client = await clientFor(c);
let before;
try { before = await client.getKeys(); }
catch (e) { throw proxyFailure(typeof e?.code === "string" && /^proxy-[a-z-]+$/.test(e.code) ? e.code : "triage-key-failed", c.purpose); }
checkSentinel(c, before);
const current = readKey(c.configDir, { uid: c.uid, fileUid: c.fileUid, purpose: c.purpose });
const quiet = current && !writes(before, current, readyPlan(c, before, current, c.now()))
&& tempsIn(keyPaths(c.configDir, c.purpose).dir).length === 0 && !fenceSet(c.stateDir);
if (quiet) {
c.hook("fast");
try {
await probeReady(client, current.key);
return { action: "held", keyId: current.keyId, rotateBy: current.rotateBy, addedKeyIds: [], removedKeyIds: [] };
} catch (e) { if (e.code !== "proxy-key-refused") throw e; }
}
c.hook("locked");
const done = await change({ ...c, client }, "ensure", (latest, record) => readyPlan(c, latest, record, c.now()));
const record = readKey(c.configDir, { uid: c.uid, fileUid: c.fileUid, purpose: c.purpose });
if (!record) throw keyError(c.purpose, "failed", "the key file is missing after the change");
await probeReady(client, record.key);
return done;
}
// The fence by name only, as keyStatus checks it: a set fence sends the launch to the lock, which
// refuses proxy-uncertain.
function fenceSet(stateDir) {
try { lstatSync(join(stateDir, "mosaic-keys.uncertain")); return true; }
catch (e) { return e.code !== "ENOENT"; }
}
// worker-key-purpose: the retire guard (reports/worker-proxy-simple.md section 2, "Retired"). True
// when an old worker key must stay: a runtime that is not ended, and not proved dead, started before
// the rotation plus marginMs, or its start is unknown. Every doubt keeps the key.
// runtimes () => listRuntimes(dataRoot) from packages/processing-control/src/runtimes.mjs.
// Missing, throwing or not an array: the key stays.
// rotatedAtMs when the current key was issued (record.issuedAt), the latest rotation. Older
// retiring keys were replaced earlier, so this is the conservative time for all.
// marginMs a launch reads the key file before it registers its runtime; a run registered up
// to marginMs after the rotation may still hold the old key.
// live, startedAt seams; the defaults read /proc.
export const RETIRE_LAUNCH_MARGIN_MS = 10 * 60_000;
export function retireHeld({ runtimes, rotatedAtMs, marginMs = RETIRE_LAUNCH_MARGIN_MS, live = runtimeLive, startedAt = runtimeStartMs } = {}) {
if (typeof runtimes !== "function" || !Number.isFinite(rotatedAtMs)) return true;
let list;
try { list = runtimes(); } catch { return true; }
if (!Array.isArray(list)) return true;
return list.some((r) => {
try {
if (r?.phase === "ended") return false;
if (live(r) === false) return false;
const started = startedAt(r);
return !Number.isFinite(started) || started <= rotatedAtMs + marginMs;
} catch { return true; }
});
}
// false only when the runtime is proved gone: its supervisor (the host process that registered it,
// the launcher that read the key) is dead or another process, and its own pid, when it has one, is
// dead too. A failed task leaves its record behind, so "not ended" alone does not mean live.
// Liveness is processing-control's own check (a pid is dead only on ESRCH or another start time),
// so an unreadable /proc keeps the key. A container runtime has no pid, and a dead docker client is
// not proof its container exited (worker-proxy.mjs finish), so it stays unknown until its
// record is ended.
const supervisorLive = (id) => (id && typeof id === "object" ? runtimeGroupState({ pid: id.pid, startTime: id.startTime, boot: id.boot }) : null);
const sameProcess = (id) => {
if (!id || !Number.isSafeInteger(id.pid) || typeof id.startTime !== "string") return null;
const now = processIdentity(id.pid);
if (!now) return false;
return now.boot === id.boot && now.startTime === id.startTime;
};
export function runtimeLive(r) {
const supervisor = supervisorLive(r?.supervisor);
if (supervisor !== false) return supervisor === true ? true : null;
if (r.kind === "container" || r.containerName !== undefined) return null;
if (r.pid === undefined) return false;
const own = runtimeGroupState(r);
return own === false ? false : own === true ? true : null;
}
// The wall-clock start of the runtime's supervisor, or null when it is not the same process on this
// boot. Start ticks since boot plus btime, at 100 ticks a second, as model-proxy's processStartMs.
export function runtimeStartMs(r) {
const id = r?.supervisor;
if (sameProcess(id) !== true) return null;
try {
const btime = Number(/^btime ([0-9]+)$/m.exec(readFileSync("/proc/stat", "utf8"))?.[1]);
const ticks = Number(id.startTime);
if (!Number.isFinite(btime) || !Number.isFinite(ticks)) return null;
return btime * 1000 + ticks * 10;
} catch { return null; }
}
// proxy-scope slice 4: op revoke, when the owner turns Triage off. Every name in this instance's
// Triage namespace leaves the proxy list, then the key file goes. No file and no names: nothing.
export async function revokeKeys(opts) {
const c = context(opts);
return change(c, "revoke", (before, current) => {
const mine = before.filter(inNamespace(c.configDir, c.purpose));
return { after: before.filter((k) => !mine.includes(k)), record: null, remove: true, done: { action: mine.length || current ? "revoked" : "none" } };
});
}
// What a sweep would do from the file alone: "rotate" when rotateBy has come, "retire" when a
// retiring key's grace plus marginMs has passed, else null. A cheap check without the lock or the
// proxy; sweepKeys decides again under the lock.
export function sweepDue(record, nowMs, marginMs = 0) {
if (!record) return null;
if (rotateDueAt(record) <= nowMs) return "rotate";
return record.retiring.some((r) => Date.parse(r.retireAt) + marginMs <= nowMs) ? "retire" : null;
}
// The bus sweep. Under the instance proxy lock it reads the file again and acts only if the action
// is still due, so two buses that wake together change the proxy once. No file: nothing, ever.
export async function sweepKeys(opts, { marginMs = 0 } = {}) {
const c = context(opts);
return change(c, "sweep", (before, current) => {
const nowMs = c.now();
const due = sweepDue(current, nowMs, marginMs);
if (due === "rotate") {
const plan = rotatePlan(c, before, current, nowMs, marginMs);
return { ...plan, op: "rotate", done: { action: "rotated", ...plan.done } };
}
if (due === "retire") {
const plan = retirePlan(c, before, current, nowMs, marginMs);
return { ...plan, op: "retire", done: { action: "retired", ...plan.done } };
}
return { after: before, record: null, done: { action: "none" } };
});
}
// Read only: the file, and when the proxy is set up, what its list holds. No lock, no write.
// The fence is checked by name only, so status writes nothing, not even the lock's identity file.
export async function keyStatus(opts) {
const { configDir, system, uid, fileUid, purpose = "triage" } = opts;
purposeOf(purpose);
const record = readKey(configDir, { uid, fileUid, purpose });
const proxy = { state: "not-set-up", fence: false, leftoverTemps: tempsIn(keyPaths(configDir, purpose).dir).length };
if (system?.proxy && Number.isInteger(system.proxy.port)) {
try { lstatSync(join(proxyStateDir(system), "mosaic-keys.uncertain")); proxy.fence = true; }
catch (e) { if (e.code !== "ENOENT") proxy.fence = true; }
try {
const client = await loadManagementClient({ configDir: proxyConfigDir(configDir, system), proxy: system.proxy });
const keys = await client.getKeys();
const prefix = sentinelPrefix(proxyConfigDir(configDir, system));
proxy.state = keys.some((k) => k.startsWith(prefix) && SECRET.test(k.slice(prefix.length))) ? "read" : "no-sentinel";
proxy.holdsKey = record ? keys.includes(record.key) : null;
proxy.retiringHeld = record ? record.retiring.map((r) => keys.includes(r.key)) : [];
proxy.orphanKeyIds = orphansOf(configDir, keys, record, purpose).map(keyIdOf);
} catch (e) { proxy.state = "unreadable"; proxy.code = typeof e?.code === "string" && /^proxy-[a-z-]+$/.test(e.code) ? e.code : "proxy-client-response"; }
}
return { record, proxy };
}
@@ -0,0 +1,50 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { mkdtempSync, mkdirSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { startBroker } from '../src/runtime.mjs';
import { checkedTransport } from '../../harness/src/bundle.mjs';
import { businessDoc, rolesCopy, systemFor, writeJson } from '../../organization/tests/helpers.mjs';
// #1873 harness-matrix: an OAuth Claude model runs on claude-code only. A stored route that puts an
// OAuth Claude model on pi is refused at launch admission with model-harness-incompatible, even when a stale
// catalogue row still says pi. A z.ai and a codex model are admitted on pi.
const roles = { chief: { definition: 'chief-of-staff', authority: { withinRole: ['decision.resolve.technical', 'decision.resolve.delivery'], crossRole: [] } }, coder: { authority: { withinRole: [], crossRole: [] } } };
async function admission(t, entry, catalogueRow) {
const root = mkdtempSync(join(tmpdir(), 'harness-matrix-')); t.after(() => rmSync(root, { recursive: true, force: true }));
const rolesDir = rolesCopy(root), doc = businessDoc(root, 'demo');
doc.plugins = { ...doc.plugins, harness: ['pi'], 'model-provider': ['cliproxy'] };
doc.launch.models['fixture-key'] = entry;
doc.roles.coder.vars = { ...doc.roles.coder.vars, harness: 'pi', model: 'fixture-key', thinking: 'low' };
writeJson(join(root, 'businesses/demo.json'), doc); mkdirSync(join(root, 'data'), { mode: 0o700 });
const modelCatalogue = { invalidate() {}, read: async () => ({ status: 'ready', notices: [], models: [{ module: 'cliproxy', available: true, ...catalogueRow }] }) };
const bus = await startBroker({ dataRoot: join(root, 'data'), businesses: { demo: { id: 'demo', human: 'jason', arbiters: { technical: 'chief', delivery: 'chief' }, roles } }, organization: { dir: root, rolesDir, system: systemFor(root) }, modelCatalogue });
t.after(() => bus.close());
const human = bus.broker.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true });
const revision = bus.broker.request(human, { verb: 'limits' }).revision;
// The launcher's own transport check refuses first; admission is the second gate, so the request
// carries the transport the provider would give with no harness named.
const transport = checkedTransport(entry);
return bus.modelAdmission({ business: 'demo', instance: 'coder', modelKey: 'fixture-key', harness: 'pi', revision, entry, provider: entry.provider, model: entry.model, account: null, route: 'primary', transport });
}
test('#1873 a stored Claude route on pi is refused model-harness-incompatible at admission, even with a stale catalogue row that says pi', async t => {
const entry = { provider: 'anthropic', model: 'claude-opus-5-5', module: 'cliproxy', upstream: 'claude' };
assert.deepEqual(await admission(t, entry, { upstream: 'claude', model: 'claude-opus-5-5', harnesses: ['claude-code', 'pi'] }), { allow: false, code: 'model-harness-incompatible' });
});
test('#1873 a Claude model from an API-key group (upstream other) is admitted on pi; one whose name an OAuth account offers is refused', async t => {
const entry = { provider: 'other', model: 'anthropic/claude-sonnet-5-5', module: 'cliproxy', upstream: 'other' };
assert.deepEqual(await admission(t, entry, { upstream: 'other', model: 'anthropic/claude-sonnet-5-5', harnesses: ['claude-code', 'pi'] }), { allow: true });
// The catalogue holds a group model to claude-code when an OAuth Claude account offers its name.
assert.deepEqual(await admission(t, entry, { upstream: 'other', model: 'anthropic/claude-sonnet-5-5', harnesses: ['claude-code'] }), { allow: false, code: 'model-harness-incompatible' });
});
test('#1873 a z.ai and a codex model are admitted on pi', async t => {
for (const [provider, upstream, model] of [['zai', 'zai', 'glm-fixture'], ['codex', 'codex', 'gpt-fixture']]) {
const entry = { provider, model, module: 'cliproxy', upstream };
assert.deepEqual(await admission(t, entry, { upstream, model, harnesses: ['claude-code', 'pi'] }), { allow: true }, provider);
}
});
@@ -95,3 +95,45 @@ test('shared generation invalidates cached catalogues and an uncertain fence mak
version++; assert.equal((await service.read('one')).status, 'ready'); assert.equal(calls, 2);
uncertain = true; assert.equal((await service.read('one')).status, 'unavailable');
});
// #1873 models-offered (A9): the owner's on/off file under the instance data root.
test('the catalogue reads the offered file from the plugin data root at every read: a write counts at once, a new model is New and off, a damaged file turns every model off, harnesses never change', async t => {
const { mkdtempSync, rmSync, writeFileSync, mkdirSync, chmodSync } = await import('node:fs');
const { tmpdir } = await import('node:os'); const { join } = await import('node:path');
const { writeOffered } = await import('../../model-proxy/src/models-offered.mjs');
const root = mkdtempSync(join(tmpdir(), 'mo-cat-')); t.after(() => rmSync(root, { recursive: true, force: true }));
let calls = 0;
const names = [{ name: 'glm-5' }];
const client = { getCredentials: async () => ({ files: [] }), getCredentialModels: async () => ({ models: [] }),
getUpstreamGroups: async () => { calls++; return [{ 'base-url': 'https://api.z.ai/api/anthropic', keys: [{}], models: [...names] }]; } };
const service = createModelCatalogue({ client: () => client, settings: () => ({ 'model-provider': ['cliproxy'] }), pluginDataRoot: root, now: () => 1000 });
const view = c => c.models.map(m => [m.model, m.available, m.reason, m.isNew ?? false, m.harnesses.join(',')]);
assert.deepEqual(view(await service.read('one')), [['glm-5', true, null, false, 'claude-code,pi']], 'no file: every model as listed');
writeOffered(root, { version: 1, providers: { 'z.ai': { on: [], seen: ['glm-5'] } } });
assert.deepEqual(view(await service.read('one')), [['glm-5', false, 'model-off', false, 'claude-code,pi']], 'a write counts at the next read, inside the cache window');
assert.equal(calls, 2);
writeOffered(root, { version: 1, providers: { 'z.ai': { on: ['glm-5'], seen: ['glm-5'] } } });
names.push({ name: 'glm-6' });
assert.deepEqual(view(await service.read('one')), [['glm-5', true, null, false, 'claude-code,pi'], ['glm-6', false, 'model-off', true, 'claude-code,pi']]);
const before = calls;
await service.read('one'); assert.equal(calls, before, 'an unchanged file keeps the cache');
const file = join(root, 'state', 'models-offered.json');
for (const [text, mode] of [['{"version":1,"providers":{"z.ai":{"on":["glm-5"],"seen":["glm-5"],"harness":"pi"}}}', 0o600], ['not json', 0o600], ['{"version":1,"providers":{"z.ai":{"on":["glm-5"],"seen":["glm-5"]}}}', 0o644]]) {
writeFileSync(file, text); chmodSync(file, mode);
const out = await service.read('one');
assert.equal(out.status, 'ready', text);
assert.deepEqual(out.models.map(m => [m.available, m.reason]), [[false, 'model-off'], [false, 'model-off']], `damaged (${text}, ${mode.toString(8)}): every model off, never all on`);
}
rmSync(file); mkdirSync(file);
assert.ok((await service.read('one')).models.every(m => m.available === false && m.reason === 'model-off'), 'a directory in place of the file is damaged');
});
test('a provider answer whose isNew is anything but absent or true makes the catalogue unavailable', async () => {
let isNew;
const model = () => ({ key: 'fixture-model', module: 'fixture-check', upstream: 'fixture', provider: 'fixture', model: 'fixture-model', label: 'Fixture', available: true, reason: null, harnesses: ['pi'], ...(isNew === undefined ? {} : { isNew }) });
const module = { catalogue: async () => ({ status: 'ready', models: [model()], notices: [] }) };
const makeLoader = () => ({ load: async () => ({ refused: [] }), snapshot: () => [{ name: 'fixture-check', module }] });
const service = createModelCatalogue({ client: () => null, settings: () => ({ 'model-provider': ['fixture-check'] }), makeLoader });
for (const good of [undefined, true]) { isNew = good; assert.equal((await service.read('one', { fresh: true })).status, 'ready', String(good)); }
for (const bad of [false, 'yes', 1, null, {}]) { isNew = bad; assert.equal((await service.read('one', { fresh: true })).status, 'unavailable', JSON.stringify(bad)); }
});
@@ -27,7 +27,9 @@ export async function ownershipFixture(t,{launches=true,listed=true,keep=false,r
// Available before any acquired resource/setup can fail; no later f/launcher TDZ.
const launchLog=()=>{const file=launchLogFile(join(root,'data'),'acme');return fs.existsSync(file)?fs.readFileSync(file,'utf8').trim().split('\n').filter(Boolean).map(line=>JSON.parse(line)):[];};
t.after(async()=>{const failures=[];for(const g of gates)g.release();try{await waitFor(()=>active.size===0,'owned operations did not settle',5000);}catch{failures.push('owned-operation-close');}for(const close of [...closers].reverse()){try{await close();}catch{failures.push('owned-close');}}try{server?.closeAllConnections();if(server?.listening)await new Promise((resolve,reject)=>server.close(e=>e?reject(e):resolve()));}catch{failures.push('owned-server-close');}try{checkHistory({logs,proxyReceipts,trace});}catch{failures.push('value-free-receipts');}try{checkHistory({launch:launchLog()});}catch{failures.push('owned-launch-log-guard');}if(stderr)process.stderr.write=stderr;try{fs.rmSync(root,{recursive:true,force:true});}catch{failures.push('owned-root-close');}if(failures.length)throw new AggregateError(failures.map(code=>Error(code)),'owned managed fixture cleanup failed');});
const f=fixture(root,'acme',doc=>{doc.plugins={...doc.plugins,harness:['pi']};for(const role of Object.values(doc.roles))role.vars={...role.vars,harness:'pi'};doc.roles.chief.vars.model='claude-opus-5-5';doc.roles.aide={definition:'aide',vars:{harness:'pi',model:'claude-opus-5-5',thinking:'high'}};if(listed)doc.launch.instances.push('aide');if(keep)doc.launch.keep=['aide'];if(retired)doc.retired=['alpha'];if(noHire)doc.launch.noHire=['aide'];if(!listed)delete doc.roles.aide;return doc;});
const f=fixture(root,'acme',doc=>{doc.plugins={...doc.plugins,harness:['pi']};for(const role of Object.values(doc.roles))role.vars={...role.vars,harness:'pi'};doc.roles.chief.vars.model='claude-opus-5-5';doc.roles.aide={definition:'aide',vars:{harness:'pi',model:'claude-opus-5-5',thinking:'high'}};if(listed)doc.launch.instances.push('aide');if(keep)doc.launch.keep=['aide'];if(retired)doc.retired=['alpha'];if(noHire)doc.launch.noHire=['aide'];if(!listed)delete doc.roles.aide;doc.launch.models['claude-opus-5-5']={provider:'zai',model:'fixture-aide-model'};return doc;});
// #1873 harness-matrix: an OAuth Claude model runs on claude-code only, so on pi the model key
// claude-opus-5-5 names a non-Claude upstream model.
// The established scratch aide pattern only; repository roles/policy stay untouched.
for(const name of ['aide','helper']){fs.writeFileSync(join(f.roles,name+'.md'),'# Scratch aide\n\n## Authority\n');writeJson(join(f.roles,name+'.json'),{roleVersion:2,name,title:'Aide',contract:name+'.md',tools:['read'],network:'api-only',authority:{withinRole:['message.send'],crossRole:[]},credentials:[]});}
const chiefFile=join(f.roles,'chief-of-staff.json');
@@ -74,6 +76,7 @@ export async function ownershipFixture(t,{launches=true,listed=true,keep=false,r
case 'landRelease':return bus.landRelease(m.record);
case 'threadsState':return bus.threadsState(m.business);
case 'threadIdle':return bus.threadIdle(m.record);
case 'failureNotice':return bus.failureNotice(m.record);
case 'lifecycleRequests':return bus.lifecycleRequests(m.business);
case 'lifecycleTake':return bus.lifecycleTake(m.record);
case 'lifecycleDelete':return bus.lifecycleDelete(m.record);
@@ -6,6 +6,7 @@ import { join } from 'node:path';
import { sandboxProblem } from '../../seat/src/sandbox.mjs';
import { startTimeOf } from '../../seat/src/proc.mjs';
import { ownershipFixture, waitFor } from './model-hire-ownership-fixture.mjs';
import * as pi from '../../plugins/harness/pi/index.mjs';
const unavailable=sandboxProblem()??false; // Any skip means UNVERIFIED, not green acceptance.
const done=async box=>{await box.promise;return box;};
const refused=async(box,code)=>{await done(box);assert.equal(box.ok,false,'real operation refuses');assert.equal(box.code,code,'real operation preserves exact refusal');};
@@ -17,7 +18,12 @@ test('655 listed duplicate with no record refuses before any model or management
assert.deepEqual(f.snapshot(),before,'duplicate hire makes no credential or management effects');f.assertHistoryGuard();f.valueFree();
});
test('655 actual launch own preflight generation and bound consumer survive concurrent duplicate hire',{skip:unavailable,timeout:30000},async t=>{
test('655 actual launch own preflight generation and bound consumer survive concurrent duplicate hire',{skip:unavailable,timeout:120000},async t=>{
// #1873 pi-unpin re-check 9 and 10: the launch's preflight shares pi's fit check, which is cold in
// a fresh process. Wait for it before the fixture starts its launcher, so the launcher's warm-up
// and the launch's preflight hit the cache, the gate's 5 s times the launch, and the launch starts
// at the same point in the launcher's 2 s tick as it did before the fit check existed.
await pi.ready({env:{PATH:process.env.PATH,HOME:process.env.HOME,TMPDIR:process.env.TMPDIR}});
const f=await ownershipFixture(t),pGet=f.hold('GET'),p=f.launchAide();await pGet.entered();
// P's own acquisition GET is held. Its PUT confirmation GET is ordinal+1;
// only an OLD H reuse GET can be ordinal+2. The correct early refusal removes it.
+11 -4
View File
@@ -80,7 +80,9 @@ function instance(t) {
// A real organization bus on the instance, started with no proxy unless `binding.proxy` gives one.
async function orgBus(t, f, id, { providers = ['cliproxy'], binding = {} } = {}) {
const doc = businessDoc(f.root, id);
doc.plugins = { harness: ['pi'], 'model-provider': providers };
// The coder's model is a Claude model on a legacy entry, which counts as OAuth and runs on
// claude-code only, so the organization enables claude-code.
doc.plugins = { harness: ['claude-code'], 'model-provider': providers };
writeJson(join(f.configDir, 'businesses', `${id}.json`), doc);
const business = loadBusiness(id, { dir: f.configDir, rolesDir: f.rolesDir }), resolved = {};
for (const name of Object.keys(business.roles)) resolved[name] = resolveInstance({ system: f.system, business, instance: name });
@@ -113,7 +115,7 @@ async function pickModel(o) {
const view = await o.ask('routes');
assert.equal(view.catalogue.status, 'ready');
assert.ok(view.catalogue.models.some(m => m.key === MODEL));
await o.ask('routes.set', { revision: view.revision, routes: { coder: { harness: 'pi', model: MODEL, thinking: 'low', fallback: null } } });
await o.ask('routes.set', { revision: view.revision, routes: { coder: { harness: 'claude-code', model: MODEL, thinking: 'low', fallback: null } } });
}
// The host launcher as cli.mjs builds it for a bus that booted with no proxy.json: its system has
@@ -123,7 +125,10 @@ async function hostLauncher(t, f, o, { system = {} } = {}) {
initializeControl(o.dataRoot);
// A launch whose role names a skill that is not enabled refuses before the model gate.
if (!fs.existsSync(join(f.dataRoot, 'skills-enabled'))) fs.cpSync(REPO_SKILLS, join(f.dataRoot, 'skills-enabled'), { recursive: true });
const env = withoutHostCredentials(humanEnv({ MOSAIC_CONFIG: join(f.configDir, 'config.json'), MOSAIC_ROLES_DIR: f.rolesDir }), { piKey: true });
// claude-code's preflight needs a session login or a proxy key, and this launcher boots with no
// proxy, so the env carries a stand-in login (as launcher.test.mjs does). No adapter ever runs.
const env = withoutHostCredentials(humanEnv({ MOSAIC_CONFIG: join(f.configDir, 'config.json'), MOSAIC_ROLES_DIR: f.rolesDir }));
env.CLAUDE_CODE_OAUTH_TOKEN = 'stand-in';
const answers = [], ops = [];
const host = { business: o.id, beforeClose: () => {}, op: async m => {
ops.push(m.op);
@@ -134,7 +139,9 @@ async function hostLauncher(t, f, o, { system = {} } = {}) {
return {};
} };
const cliproxy = JSON.stringify(pathToFileURL(new URL('../../plugins/model-provider/cliproxy/index.mjs', import.meta.url).pathname).href);
const makeLoader = await fakeHarnesses(t, { pi: null }, write => write('model-provider', 'cliproxy', `export * from ${cliproxy};\n`));
// The stand-in adapter path skips only the binary check; the launch stops at the model gate or
// authorizeLaunch, before any adapter runs.
const makeLoader = await fakeHarnesses(t, { 'claude-code': '/fixture-unused' }, write => write('model-provider', 'cliproxy', `export * from ${cliproxy};\n`));
// The validated system config shape (mosaic-config.mjs), as cli.mjs passes it with the org root.
const config = { configVersion: 1, environment: 'development', dataRoot: o.dataRoot, instanceRoot: f.dataRoot,
execution: { backend: 'docker', provider: 'zai', model: 'glm-5.3-flash', adapter: 'mock' }, ...system };
+64
View File
@@ -102,3 +102,67 @@ test('plugins read advertises loadable harnesses even when switched off', async
const view=host.broker.request(human,{verb:'plugins'});
assert.deepEqual(view.plugins.harness,['claude-code']);assert.ok(view.available.harness.includes('pi'));assert.ok(view.available.harness.includes('claude-code'));
});
// #1873 harness watch W3: the plugins answer carries what the host's watch last wrote under the
// organization's data root (state/harness-watch.json), read only, redacted again, and "not checked
// yet" for every harness when the file is missing, cut short or not version 1's shape.
test('W3 runtime: the plugins answer carries the harness watch record per harness, and a missing, truncated or wrong-shape file gives not checked yet', async (t) => {
const { startBroker } = await import('../src/runtime.mjs');
const { businessDoc, rolesCopy, systemFor, writeJson } = await import('../../organization/tests/helpers.mjs');
const { writeFileSync } = await import('node:fs');
const root = mkdtempSync(join(tmpdir(), 'plugins-watch-'));
const rolesDir = rolesCopy(root);
writeJson(join(root, 'businesses/demo.json'), businessDoc(root, 'demo'));
const dataRoot = join(root, 'data');
mkdirSync(join(dataRoot, 'state'), { recursive: true, mode: 0o700 });
const planted = 'w3-planted-env-value';
process.env.W3_PLANTED = planted;
const bus = await startBroker({ dataRoot, businesses: { demo: { id: 'demo', human: 'jason', arbiters: { technical: 'chief', delivery: 'chief' }, roles: { chief: { definition: 'chief-of-staff', authority: { withinRole: ['decision.resolve.technical', 'decision.resolve.delivery'], crossRole: [] } }, coder: { authority: { withinRole: [], crossRole: [] } } } } }, organization: { dir: root, rolesDir, system: systemFor(root) } });
t.after(async () => { delete process.env.W3_PLANTED; await bus.close(); rmSync(root, { recursive: true, force: true }); });
const human = bus.broker.bindHuman({ business: 'demo', human: 'jason', via: 'cli', outsideAgent: true });
const file = join(dataRoot, 'state', 'harness-watch.json');
const read = () => bus.broker.request(human, { verb: 'plugins' });
const names = read().available.harness;
assert.ok(names.length > 0, 'the tree gives harnesses');
const unchecked = (name) => ({ name, state: 'unchecked', version: null, checkedAt: null, changedAt: null, failed: [], line: 'not checked yet' });
// No file: every harness the loader gives reads not checked yet, in the loader's order.
assert.deepEqual(read().harnessWatch, names.map(unchecked));
const at = '2026-11-20T10:00:00.000Z', moved = '2026-11-19T08:00:00.000Z';
const rec = (harness, over) => ({ harness, state: 'fit', via: 'inspect', found: true, source: 'PATH', version: '1.0.0', digest: 'a'.repeat(64), lastDigest: 'a'.repeat(64), fit: 'pass', definite: true, failed: [], line: 'ok', checkedAt: at, changedAt: moved, ...over });
const doc = { version: 1, checkedAt: at, refused: [], gone: [], harnesses: [
rec(names[0], { version: '2.1.323', line: 'every flag the adapter passes is in --help' }),
rec('w3-red', { state: 'red', fit: 'fail', failed: ['--restricted'], line: '--help lacks --restricted' }),
rec('w3-missing', { state: 'missing', found: false, version: null, digest: null, fit: 'unknown', definite: false, line: 'no binary from MOSAIC_PI_BIN, PATH or the repo copy' }),
rec('w3-unknown', { state: 'unknown', fit: 'unknown', definite: false, line: 'inspect did not answer in time' }),
// A path and an env value planted in the file: the broker redacts them again before answering.
rec('w3-leak', { state: 'red', fit: 'fail', version: `9.9 ${planted}`, failed: [planted, '--ok'], line: `failed under ${dataRoot}/x/y and ${join(root, 'businesses')}/demo.json with ${planted}` }),
] };
writeFileSync(file, JSON.stringify(doc), { mode: 0o600 });
const view = read().harnessWatch;
const by = Object.fromEntries(view.map((e) => [e.name, e]));
assert.deepEqual(view.map((e) => e.name), [...names, 'w3-red', 'w3-missing', 'w3-unknown', 'w3-leak'], 'the loader\'s harnesses first, then the file\'s others');
assert.deepEqual(by[names[0]], { name: names[0], state: 'fit', version: '2.1.323', checkedAt: at, changedAt: moved, failed: [], line: 'every flag the adapter passes is in --help' });
assert.deepEqual(by['w3-red'], { name: 'w3-red', state: 'red', version: '1.0.0', checkedAt: at, changedAt: moved, failed: ['--restricted'], line: '--help lacks --restricted' });
assert.deepEqual(by['w3-missing'], { name: 'w3-missing', state: 'missing', version: null, checkedAt: at, changedAt: moved, failed: [], line: 'no binary from MOSAIC_PI_BIN, PATH or the repo copy' });
assert.deepEqual(by['w3-unknown'], { name: 'w3-unknown', state: 'unknown', version: '1.0.0', checkedAt: at, changedAt: moved, failed: [], line: 'inspect did not answer in time' });
for (const name of names.slice(1)) assert.deepEqual(by[name], unchecked(name), 'a harness with no record is not checked yet');
const leak = JSON.stringify(by['w3-leak']);
for (const needle of [planted, dataRoot, root]) assert.ok(!leak.includes(needle), `the answer holds no ${needle === planted ? 'env value' : 'host path'}: ${leak}`);
assert.deepEqual(by['w3-leak'].failed, ['--ok'], 'a rule id that is an env value is dropped');
assert.match(by['w3-leak'].line, /^failed under <path>.* with <W3_PLANTED>$/);
assert.deepEqual(JSON.parse(readFileSync(file, 'utf8')), doc, 'the broker does not write the file');
// A truncated file, a wrong shape and a wrong version each read as not checked yet, never a throw.
for (const bad of [JSON.stringify(doc).slice(0, 80), JSON.stringify({ ...doc, harnesses: { [names[0]]: doc.harnesses[0] } }), JSON.stringify({ ...doc, version: 2 }), JSON.stringify([doc]), 'null', '']) {
writeFileSync(file, bad, { mode: 0o600 });
assert.deepEqual(read().harnessWatch, names.map(unchecked), bad.slice(0, 40));
}
// A record of the wrong shape reads as not checked yet for that harness only.
writeFileSync(file, JSON.stringify({ ...doc, harnesses: [rec(names[0], { state: 'green' }), rec('w3-red', { state: 'red', fit: 'fail' }), 'x', null, rec('../x', {})] }), { mode: 0o600 });
const odd = Object.fromEntries(read().harnessWatch.map((e) => [e.name, e]));
assert.deepEqual(odd[names[0]], unchecked(names[0]));
assert.equal(odd['w3-red'].state, 'red');
assert.equal(odd['../x'], undefined, 'a name that is not a harness name is dropped');
});
+105 -1
View File
@@ -4,6 +4,7 @@ import test from 'node:test';
import assert from 'node:assert/strict';
import { createHash } from 'node:crypto';
import { fakeManagement } from './proxy-page-fixture.mjs';
import { PROVIDERS } from '../../plugins/model-provider/cliproxy/index.mjs';
const id = value => createHash('sha256').update(value).digest('hex').slice(0, 12);
const oauth = extra => ({ id: 'oauth-one.json', name: 'oauth-one.json', provider: 'claude', label: 'p4-private-label',
email: 'p4-private-email', token: 'p4-private-token', status: 'ready', priority: 3, ...extra });
@@ -11,7 +12,9 @@ test('accounts have exactly the allowlisted public fields, id from credential id
const s = await fakeManagement(t, { files: [oauth()] });
const result = await s.call('proxy.accounts');
assert.deepEqual(result, { accounts: [{ accountId: id('oauth-one.json'), provider: 'claude',
displayName: `Claude account ${id('oauth-one.json')}`, status: 'ready', priority: 3 }] });
displayName: `Claude account ${id('oauth-one.json')}`, status: 'ready', priority: 3 }],
// settings-backend-a3: the provider list rides along, id, label and sign-in kind only.
providers: PROVIDERS.map(({ id, label, signIn }) => ({ id, label, signIn })) });
for (const value of ['p4-private-label', 'p4-private-email', 'p4-private-token']) {
assert.equal(JSON.stringify(result).includes(value), false);
assert.notEqual(result.accounts[0].accountId, id(value));
@@ -140,3 +143,104 @@ test('OAuth callback changes the shared generation for other organization catalo
await s.call('proxy.login.submit', { redirect_url: `http://localhost/?state=${login.state}&code=fixture` });
assert.equal(lock.generation(), 1);
});
// #1873 account-remove (Jason 2026-11-20): proxy.account.remove { accountId, confirm: true }.
const ZAI = 'https://api.z.ai/api/anthropic';
const zaiGroup = (keys, extra = {}) => ({ name: 'fixture-zai', 'base-url': ZAI, models: [{ name: 'glm-fixture' }], ...extra,
keys: keys.map(k => ({ 'api-key': `${k}-secret`, auth_index: k })) });
test('remove: an OAuth account is deleted by its credential name, read back gone, and the catalogue told', async t => {
const changes = [];
const s = await fakeManagement(t, { files: [oauth(), oauth({ id: 'oauth-two.json', name: 'oauth-two.json' })], onChange: c => changes.push(c) });
const result = await s.call('proxy.account.remove', { accountId: id('oauth-one.json'), confirm: true });
assert.deepEqual(result, { accountId: id('oauth-one.json'), provider: 'claude', removed: true });
assert.deepEqual(s.state.deleted, ['oauth-one.json']);
assert.deepEqual((await s.call('proxy.accounts')).accounts.map(a => a.accountId), [id('oauth-two.json')]);
assert.ok(changes.includes('invalidate'));
for (const value of ['p4-private-label', 'p4-private-email', 'p4-private-token', 'oauth-one.json']) {
assert.equal(JSON.stringify(result).includes(value), false);
assert.equal(JSON.stringify(s.state.receipts).includes(value), false);
}
});
test('remove: a z.ai group with one key goes; with two keys only that key goes and the rest of the group stays', async t => {
const s = await fakeManagement(t, { groups: [zaiGroup(['zai-one']), zaiGroup(['zai-two', 'zai-three'], { name: 'fixture-zai-2' })] });
assert.deepEqual(await s.call('proxy.account.remove', { accountId: id('zai-one'), confirm: true }), { accountId: id('zai-one'), provider: 'z.ai', removed: true });
assert.deepEqual(s.state.groups.map(g => g.name), ['fixture-zai-2']);
await s.call('proxy.account.remove', { accountId: id('zai-two'), confirm: true });
assert.deepEqual(s.state.groups.map(g => [g.name, g.keys.map(k => k['api-key']), g.models]), [['fixture-zai-2', ['zai-three-secret'], [{ name: 'glm-fixture' }]]]);
assert.equal(s.state.calls.some(c => c.startsWith('DELETE ')), false);
for (const value of ['zai-one-secret', 'zai-two-secret']) assert.equal(JSON.stringify(s.state.receipts).includes(value), false);
});
test('remove-unconfirmed: the proxy answers ok but the account is still listed', async t => {
const s = await fakeManagement(t, { files: [oauth()], deleteNoop: true });
await assert.rejects(s.call('proxy.account.remove', { accountId: id('oauth-one.json'), confirm: true }), { code: 'remove-unconfirmed' });
const g = await fakeManagement(t, { groups: [zaiGroup(['zai-one'])], groupsNoop: true });
await assert.rejects(g.call('proxy.account.remove', { accountId: id('zai-one'), confirm: true }), { code: 'remove-unconfirmed' });
assert.equal(g.state.receives, 1);
});
test('remove-write-refused: a 500 from the proxy, a partial delete, or a refused group write', async t => {
const a = await fakeManagement(t, { files: [oauth()], deleteStatus: 500 });
await assert.rejects(a.call('proxy.account.remove', { accountId: id('oauth-one.json'), confirm: true }), { code: 'remove-write-refused' });
const b = await fakeManagement(t, { files: [oauth()], deleteBody: { status: 'partial', deleted: 0, failed: [{ name: 'oauth-one.json' }] }, deleteNoop: true });
await assert.rejects(b.call('proxy.account.remove', { accountId: id('oauth-one.json'), confirm: true }), { code: 'remove-write-refused' });
const c = await fakeManagement(t, { groups: [zaiGroup(['zai-one'])], groupsPutStatus: 500 });
await assert.rejects(c.call('proxy.account.remove', { accountId: id('zai-one'), confirm: true }), { code: 'remove-write-refused' });
assert.equal(c.state.groups.length, 1);
});
test('remove: a missing or false confirm, an unknown or malformed id, or an extra field refuses before any write', async t => {
const s = await fakeManagement(t, { files: [oauth()] });
for (const args of [{ accountId: id('oauth-one.json') }, { accountId: id('oauth-one.json'), confirm: false }, { accountId: id('oauth-one.json'), confirm: 'true' },
{ accountId: 'a'.repeat(12), confirm: true }, { accountId: 'oauth-one.json', confirm: true }, { accountId: id('oauth-one.json').toUpperCase(), confirm: true },
{ accountId: id('oauth-one.json'), confirm: true, name: 'oauth-one.json' }])
await assert.rejects(s.call('proxy.account.remove', args), { code: 'invalid-request' });
assert.equal(s.state.calls.some(c => !c.startsWith('GET ')), false);
});
test('remove: a fenced shared proxy refuses with proxy-uncertain and writes nothing', async t => {
const fs = await import('node:fs'); const { join } = await import('node:path'); const { tmpdir } = await import('node:os');
const { createProxyLock } = await import('../src/proxy-lock.mjs');
const dir = fs.mkdtempSync(join(tmpdir(), 'p4-remove-')); t.after(() => fs.rmSync(dir, { recursive: true, force: true }));
const lock = createProxyLock({ stateDir: dir, identity: 'fixture' });
const s = await fakeManagement(t, { lock, org: 'alpha', files: [oauth()], groups: [zaiGroup(['zai-one'])] });
await lock.run(async () => lock.fence({ org: 'beta', op: 'keys.put' }));
await assert.rejects(s.call('proxy.account.remove', { accountId: id('oauth-one.json'), confirm: true }), { code: 'proxy-uncertain' });
await assert.rejects(s.call('proxy.account.remove', { accountId: id('zai-one'), confirm: true }), { code: 'proxy-uncertain' });
assert.equal(s.state.calls.some(c => !c.startsWith('GET ')), false);
assert.equal(s.state.files.length, 1);
});
test('remove: under the lock, a delete that fails after the fence leaves the proxy uncertain', async t => {
const fs = await import('node:fs'); const { join } = await import('node:path'); const { tmpdir } = await import('node:os');
const { createProxyLock } = await import('../src/proxy-lock.mjs');
const dir = fs.mkdtempSync(join(tmpdir(), 'p4-remove-')); t.after(() => fs.rmSync(dir, { recursive: true, force: true }));
const lock = createProxyLock({ stateDir: dir, identity: 'fixture' });
const s = await fakeManagement(t, { lock, org: 'alpha', files: [oauth()], deleteStatus: 500 });
await assert.rejects(s.call('proxy.account.remove', { accountId: id('oauth-one.json'), confirm: true }), { code: 'proxy-uncertain' });
});
test('remove: two listed accounts on one credential name, or one account id listed twice, refuse before any write', async t => {
// The DELETE names the file, so it would take the second account too while reporting one removed.
const s = await fakeManagement(t, { files: [oauth(), oauth({ id: 'oauth-other-id', name: 'oauth-one.json' })] });
await assert.rejects(s.call('proxy.account.remove', { accountId: id('oauth-one.json'), confirm: true }), { code: 'invalid-request' });
await assert.rejects(s.call('proxy.account.remove', { accountId: id('oauth-other-id'), confirm: true }), { code: 'invalid-request' });
const d = await fakeManagement(t, { files: [oauth(), oauth({ name: 'oauth-one-copy.json' })] });
await assert.rejects(d.call('proxy.account.remove', { accountId: id('oauth-one.json'), confirm: true }), { code: 'invalid-request' });
for (const x of [s, d]) { assert.equal(x.state.calls.some(c => !c.startsWith('GET ')), false); assert.equal(x.state.files.length, 2); }
});
test('remove: a credential with an unusable id still prevents deleting its shared name', async t => {
for (const badId of ['', null, 'x'.repeat(513)]) {
const s = await fakeManagement(t, { files: [oauth(), oauth({ id: badId, name: 'oauth-one.json' })] });
await assert.rejects(s.call('proxy.account.remove', { accountId: id('oauth-one.json'), confirm: true }), { code: 'invalid-request' });
assert.equal(s.state.calls.some(c => !c.startsWith('GET ')), false);
assert.equal(s.state.files.length, 2);
}
});
test('remove: under the lock, key groups that change between the read and the write refuse with proxy-down before the fence', async t => {
const fs = await import('node:fs'); const { join } = await import('node:path'); const { tmpdir } = await import('node:os');
const { createProxyLock } = await import('../src/proxy-lock.mjs');
const dir = fs.mkdtempSync(join(tmpdir(), 'p4-remove-')); t.after(() => fs.rmSync(dir, { recursive: true, force: true }));
const lock = createProxyLock({ stateDir: dir, identity: 'fixture' });
const s = await fakeManagement(t, { lock, org: 'alpha', files: [oauth()], groups: [zaiGroup(['zai-one'])] });
// Another writer adds a key group after the page's first read.
const read = s.client.getUpstreamGroups.bind(s.client); let reads = 0;
s.client.getUpstreamGroups = async () => { const groups = await read(); return ++reads === 2 ? [...groups, zaiGroup(['zai-late'], { name: 'fixture-late' })] : groups; };
await assert.rejects(s.call('proxy.account.remove', { accountId: id('oauth-one.json'), confirm: true }), { code: 'proxy-down' });
assert.equal(s.state.calls.some(c => !c.startsWith('GET ')), false);
assert.equal(lock.generation(), 0, 'no fence was set');
});
+11 -2
View File
@@ -14,6 +14,14 @@ export async function fakeManagement(t, options = {}) {
state.calls.push(req.method + ' ' + u.pathname);
const reply = (status, body) => { res.writeHead(status, { 'content-type': 'application/json' }); res.end(JSON.stringify(body)); };
if (req.headers.authorization !== 'Bearer p4-management') return reply(401, { error: 'refused' });
// #1873 account-remove: DELETE takes {names:[...]} (DeleteAuthFileV8). deleteNoop answers ok and
// keeps the file; deleteStatus and deleteBody shape a refusal.
if (u.pathname === '/v8/management/credentials' && req.method === 'DELETE') {
state.deleted = [...(state.deleted ?? []), ...(value?.names ?? [])];
if ((state.deleteStatus ?? 200) !== 200) return reply(state.deleteStatus, { error: 'p4-private-detail' });
if (!state.deleteNoop) state.files = state.files.filter(f => !(value?.names ?? []).includes(f.name));
return reply(200, state.deleteBody ?? { status: 'ok', deleted: (value?.names ?? []).length, failed: [] });
}
if (u.pathname === '/v8/management/credentials') return reply(200, { files: state.files, observed_at: '2026-10-24T00:00:00Z' });
if (u.pathname === '/v8/management/config/api-keys/claude') {
if (req.method === 'GET') {
@@ -24,7 +32,8 @@ export async function fakeManagement(t, options = {}) {
// Stand-in treats auth_index as derived/read-only metadata.
if (value.some(g => (g.keys ?? []).some(k => Object.hasOwn(k, 'auth_index')))) return reply(422, { error: 'derived field' });
state.receives++;
if (!state.priorityNoop) state.groups = value.map(g => ({ ...g, keys: g.keys.map(k => ({ ...k, auth_index: state.groups.flatMap(old => old.keys ?? []).find(old => old['api-key'] === k['api-key'])?.auth_index ?? `upstream-${nextId++}` })) }));
if (state.groupsPutStatus) return reply(state.groupsPutStatus, { error: 'p4-private-detail' });
if (!state.priorityNoop && !state.groupsNoop) state.groups = value.map(g => ({ ...g, keys: g.keys.map(k => ({ ...k, auth_index: state.groups.flatMap(old => old.keys ?? []).find(old => old['api-key'] === k['api-key'])?.auth_index ?? `upstream-${nextId++}` })) }));
return reply(200, { status: 'ok' });
}
if (u.pathname === '/v8/management/credentials/fields') {
@@ -43,7 +52,7 @@ export async function fakeManagement(t, options = {}) {
await new Promise(r => server.listen(0, '127.0.0.1', r));
t.after(() => new Promise(r => server.close(r)));
const client = new ManagementClient({ baseUrl: `http://127.0.0.1:${server.address().port}`, key: 'p4-management', receipt: entry => state.receipts.push(entry) });
const page = new ProxyPage({ client, instanceTag: 'fixture', lock: options.lock ?? null, org: options.org ?? null, keyRecords: () => options.records ?? [] });
const page = new ProxyPage({ client, instanceTag: 'fixture', lock: options.lock ?? null, org: options.org ?? null, keyRecords: () => options.records ?? [], onChange: options.onChange ?? (() => {}) });
const call = (verb, args = {}) => page.request(verb, args, { business: 'demo', human: 'owner' });
return { state, page, call, client, server };
}
@@ -0,0 +1,106 @@
// #1873 settings-backend-a3: the proxy page's provider names, login allow list, sign-in host and key
// allow list all come from the plugin's PROVIDERS. A provider a test copy of the list leaves out is
// refused at each door with the same code as before; one a copy adds is let through. The accounts
// answer carries the list as id, label and sign-in kind only.
import test from 'node:test';
import assert from 'node:assert/strict';
import { ProxyPage } from '../src/proxy-page.mjs';
import { PROVIDERS } from '../../plugins/model-provider/cliproxy/index.mjs';
const owner = { business: 'demo', human: 'owner' };
function setup(options = {}) {
const calls = [], state = { files: [], groups: [] };
const client = Object.fromEntries(['getCredentials', 'oauthStart', 'oauthCallback', 'oauthStatus', 'oauthCancel', 'getUpstreamGroups', 'putUpstreamGroups'].map(name => [name, async (...args) => {
calls.push([name, ...args]);
if (name === 'oauthStart') return { status: 'ok', state: `s-${calls.length}`, url: `https://${options.hosts?.[args[0]] ?? 'nowhere.invalid'}/oauth/authorize?state=s-${calls.length}` };
if (name === 'getCredentials') return { files: state.files };
if (name === 'getUpstreamGroups') return structuredClone(state.groups);
if (name === 'putUpstreamGroups') { state.groups = args[0].map((g, i) => ({ ...g, keys: g.keys.map((k, j) => ({ ...k, auth_index: `idx-${i}-${j}` })) })); return { status: 'ok' }; }
return { status: 'ok' };
}]));
const p = new ProxyPage({ client, instanceTag: 'fixture', ...(options.providers ? { providers: options.providers } : {}) });
return { calls, state, call: (verb, args = {}) => p.request(verb, args, owner) };
}
const hosts = Object.fromEntries(PROVIDERS.filter(p => p.signIn === 'oauth').map(p => [p.id, p.signInHost]));
const door = p => p.signIn === 'oauth' ? ['proxy.login.start', { provider: p.id }] : ['proxy.apikey.add', { provider: p.id, key: 'fixture-key' }];
const otherDoor = p => p.signIn === 'oauth' ? ['proxy.apikey.add', { provider: p.id, key: 'fixture-key' }] : ['proxy.login.start', { provider: p.id }];
test('a3: each provider in PROVIDERS passes its own door and is refused at the other with provider-not-allowed', async () => {
assert.ok(PROVIDERS.some(p => p.signIn === 'oauth') && PROVIDERS.some(p => p.signIn === 'apikey'));
for (const p of PROVIDERS) {
const s = setup({ hosts });
const answer = await s.call(...door(p));
if (p.signIn === 'oauth') assert.equal(new URL(answer.url).hostname, p.signInHost, p.id);
else { assert.deepEqual(answer, { added: true }); assert.equal(s.state.groups.at(-1)['base-url'], p.baseUrl); }
const before = s.calls.length;
await assert.rejects(s.call(...otherDoor(p)), { code: 'provider-not-allowed' }, p.id);
assert.equal(s.calls.length, before, `${p.id}: a refusal reaches no proxy call`);
}
for (const [verb, args] of [['proxy.login.start', { provider: 'other' }], ['proxy.apikey.add', { provider: 'other', key: 'fixture-key' }]]) {
const s = setup({ hosts });
await assert.rejects(s.call(verb, args), { code: 'provider-not-allowed' }); assert.equal(s.calls.length, 0);
}
});
test('a3: a provider taken out of a test copy of the list is refused at its door, reads Other and leaves the providers answer', async () => {
for (const gone of PROVIDERS) {
const s = setup({ hosts, providers: PROVIDERS.filter(p => p !== gone) });
await assert.rejects(s.call(...door(gone)), { code: 'provider-not-allowed' }, gone.id);
assert.equal(s.calls.length, 0, `${gone.id}: refused before any proxy call`);
if (gone.signIn === 'oauth') s.state.files = [{ id: 'one.json', name: 'one.json', provider: gone.id, status: 'ready' }];
else s.state.groups = [{ name: 'g', 'base-url': gone.baseUrl, keys: [{ auth_index: 'idx-0-0' }], models: [{ name: 'm1' }] }];
const answer = await s.call('proxy.accounts');
assert.equal(answer.accounts.length, 1);
assert.equal(answer.accounts[0].provider, 'other', gone.id);
assert.match(answer.accounts[0].displayName, /^Other account /);
assert.equal(Object.hasOwn(answer.accounts[0], 'models'), false, `${gone.id}: an unlisted group shows no models`);
assert.deepEqual(answer.providers.map(p => p.id), PROVIDERS.filter(p => p !== gone).map(p => p.id));
}
});
test('a3: a provider a test copy adds is let through: key group from its base URL, sign-in on its host, name from its label', async () => {
const extra = [{ id: 'fixture.ai', label: 'Fixture AI', signIn: 'apikey', baseUrl: 'https://api.fixture.invalid/v1' },
{ id: 'fixture-sso', label: 'Fixture SSO', signIn: 'oauth', signInHost: 'sso.fixture.invalid' }];
const s = setup({ providers: [...PROVIDERS, ...extra], hosts: { ...hosts, 'fixture-sso': 'sso.fixture.invalid' } });
assert.deepEqual(await s.call('proxy.apikey.add', { provider: 'fixture.ai', key: 'fixture-key' }), { added: true });
assert.equal(s.state.groups[0]['base-url'], 'https://api.fixture.invalid/v1');
assert.match(s.state.groups[0].name, /^mosaic-fixtureai-[0-9a-f]{12}$/);
assert.equal(new URL((await s.call('proxy.login.start', { provider: 'fixture-sso' })).url).hostname, 'sso.fixture.invalid');
s.state.files = [{ id: 'sso.json', name: 'sso.json', provider: 'fixture-sso', status: 'ready' }];
const { accounts } = await s.call('proxy.accounts');
assert.deepEqual(accounts.map(a => [a.provider, a.displayName.replace(/ [0-9a-f]{12}$/, '')]), [['fixture-sso', 'Fixture SSO account'], ['fixture.ai', 'Fixture AI account']]);
assert.deepEqual(accounts[1].models, []);
});
test('a3: the sign-in address must be on the host the list names for that provider', async () => {
const codex = PROVIDERS.find(p => p.id === 'codex');
const moved = PROVIDERS.map(p => p === codex ? { ...p, signInHost: 'elsewhere.invalid' } : p);
const s = setup({ hosts, providers: moved });
await assert.rejects(s.call('proxy.login.start', { provider: 'codex' }), { code: 'proxy-down' });
const t = setup({ hosts: { ...hosts, codex: 'elsewhere.invalid' }, providers: moved });
assert.equal(new URL((await t.call('proxy.login.start', { provider: 'codex' })).url).hostname, 'elsewhere.invalid');
});
test('a3: the accounts answer carries the providers as id, label and sign-in kind only, no URL, host or key', async () => {
const s = setup({ hosts });
await s.call('proxy.apikey.add', { provider: 'z.ai', key: 'a3-canary-key-value' });
const answer = await s.call('proxy.accounts');
assert.deepEqual(answer.providers, PROVIDERS.map(({ id, label, signIn }) => ({ id, label, signIn })));
for (const p of answer.providers) assert.deepEqual(Object.keys(p).sort(), ['id', 'label', 'signIn']);
const text = JSON.stringify(answer.providers);
for (const word of ['a3-canary-key-value', 'https://', 'api.z.ai', 'claude.ai', 'auth.openai.com', 'baseUrl', 'signInHost']) assert.equal(text.includes(word), false, word);
});
// Review P3-1: an entry missing what its door needs is refused before any proxy call, so no sign-in
// is opened at the proxy that the page could not cancel; a key group with no base URL is no provider's.
test('a3: an entry with no sign-in host or no key base URL is refused before any proxy call and names no group', async () => {
const bare = [{ id: 'nohost', label: 'No Host', signIn: 'oauth' }, { id: 'nourl', label: 'No URL', signIn: 'apikey' }];
const s = setup({ providers: [...PROVIDERS, ...bare], hosts: { ...hosts, nohost: 'nohost.invalid' } });
await assert.rejects(s.call('proxy.login.start', { provider: 'nohost' }), { code: 'provider-not-allowed' });
await assert.rejects(s.call('proxy.apikey.add', { provider: 'nourl', key: 'fixture-key' }), { code: 'provider-not-allowed' });
assert.equal(s.calls.length, 0);
s.state.groups = [{ name: 'g', keys: [{ auth_index: 'idx-0-0' }], models: [{ name: 'm1' }] }];
s.state.files = [{ id: 'one.json', name: 'one.json', provider: 'nohost', status: 'ready' }];
const { accounts } = await s.call('proxy.accounts');
assert.deepEqual(accounts.map(a => a.provider), ['other', 'other']);
assert.equal(accounts.some(a => Object.hasOwn(a, 'models')), false);
});
@@ -0,0 +1,82 @@
// #1873 chat-fail F3: an account the proxy reports as not working carries one plain line,
// built only from the structured fields of the proxy's own account read. No upstream text,
// label or email is ever reflected, and no new proxy call is made.
import test from 'node:test';
import assert from 'node:assert/strict';
import { fakeManagement } from './proxy-page-fixture.mjs';
const PRIVATE = ['p4-private-label', 'p4-private-email', 'p4-private-token', 'p4-private-message'];
const oauth = extra => ({ id: 'oauth-one.json', name: 'oauth-one.json', provider: 'claude', label: PRIVATE[0],
email: PRIVATE[1], token: PRIVATE[2], status: 'ready', priority: 3, ...extra });
const SIGN_IN = 'The account needs to sign in again. Sign in again, or add another account.';
const USAGE = 'The account is out of usage. Wait until it resets, or add another account.';
const usageAt = time => `The account is out of usage. It resets at ${time}. Wait until then, or add another account.`;
const PAUSED = time => `The model service paused this account until ${time}. Wait until then, sign in again, or add another account.`;
const NO_REASON = 'The model service gave no reason. Wait and try again later, sign in again, or add another account.';
async function reasonOf(t, extra) {
const s = await fakeManagement(t, { files: [oauth(extra)] });
const result = await s.call('proxy.accounts');
const text = JSON.stringify(result);
for (const value of PRIVATE) assert.equal(text.includes(value), false, `${value} reached the page`);
assert.deepEqual(s.state.calls, ['GET /v8/management/credentials', 'GET /v8/management/config/api-keys/claude'], 'no new proxy call');
return result.accounts[0];
}
test('an account that must sign in again says so, from requires_sign_in or a refresh error', async t => {
for (const extra of [{ status: 'error', requires_sign_in: true, next_action: 'sign_in', recovery_hint: PRIVATE[3] },
{ status: 'error', refresh_error: { code: 'invalid_grant', message: PRIVATE[3], http_status: 400 } },
{ unavailable: true, status_message: `401 Unauthorized ${PRIVATE[3]}` },
{ status: 'error', status_message: `token expired ${PRIVATE[3]}` }])
assert.equal((await reasonOf(t, extra)).reason, SIGN_IN, JSON.stringify(extra));
});
test('an account out of usage names the reset time when the read carries one', async t => {
assert.equal((await reasonOf(t, { unavailable: true, quota_blocked: true, next_retry_after: '2099-03-04T05:06:07Z' })).reason,
usageAt('2099-03-04 05:06 UTC'));
assert.equal((await reasonOf(t, { status: 'error', status_message: `usage limit reached ${PRIVATE[3]}`, next_retry_after: '2099-03-04T05:06:07+02:00' })).reason,
usageAt('2099-03-04 03:06 UTC'));
// No time, a past time or a value that is not a time gives no time, never the value itself.
for (const next_retry_after of [undefined, '2000-01-01T00:00:00Z', PRIVATE[3], '2099-13-40T99:99:99Z', 12345, { at: 1 }])
assert.equal((await reasonOf(t, { unavailable: true, quota_blocked: true, next_retry_after })).reason, USAGE, String(next_retry_after));
});
test('a paused account with only a retry time says until when', async t => {
assert.equal((await reasonOf(t, { unavailable: true, status_message: PRIVATE[3], next_retry_after: '2099-03-04T05:06:07.123456789Z' })).reason,
PAUSED('2099-03-04 05:06 UTC'));
});
test('a failing account with nothing more in the read says the model service gave no reason', async t => {
for (const extra of [{ status: 'error' }, { unavailable: true }, { status: 'unavailable', status_message: PRIVATE[3] },
{ status: 'error', status_message: 42, refresh_error: null, requires_sign_in: 'yes', quota_blocked: 1 }])
assert.equal((await reasonOf(t, extra)).reason, NO_REASON, JSON.stringify(extra));
});
test('a working, turned-off or unknown account keeps exactly its fields and gets no line', async t => {
for (const extra of [{}, { status: 'active', status_message: 'usage limit reached' }, { disabled: true, requires_sign_in: true }, { status: 'mystery' }])
assert.deepEqual(Object.keys(await reasonOf(t, extra)).sort(), ['accountId', 'displayName', 'priority', 'provider', 'status'], JSON.stringify(extra));
});
test('every line is plain: no em dash, no control character, at most one sentence pair', () => {
for (const line of [SIGN_IN, USAGE, usageAt('2099-03-04 05:06 UTC'), PAUSED('2099-03-04 05:06 UTC'), NO_REASON]) {
assert.doesNotMatch(line, /[—\u0000-\u001f\u007f]/);
assert.ok(line.length <= 160, line);
}
});
// Review round 1: a day the month doesn't have is refused, not rolled into the next month.
test('an impossible date gives no reset time; a real leap day does', async t => {
for (const next_retry_after of ['2099-02-31T00:00:00Z', '2099-04-31T10:00:00Z', '2099-02-29T00:00:00Z'])
assert.equal((await reasonOf(t, { unavailable: true, quota_blocked: true, next_retry_after })).reason, USAGE, next_retry_after);
assert.equal((await reasonOf(t, { unavailable: true, quota_blocked: true, next_retry_after: '2096-02-29T01:02:03Z' })).reason, usageAt('2096-02-29 01:02 UTC'));
});
// Review round 1: quota_blocked wins over a message word, and a bare "expired" is not a sign-in.
test('an account blocked on usage whose message says expired gets the usage line', async t => {
assert.equal((await reasonOf(t, { unavailable: true, quota_blocked: true, status_message: `usage window expired ${PRIVATE[3]}` })).reason, USAGE);
assert.equal((await reasonOf(t, { status: 'error', status_message: `trial window expired ${PRIVATE[3]}` })).reason, NO_REASON);
for (const status_message of ['session expired', 'credentials expired', 'OAuth token has expired'])
assert.equal((await reasonOf(t, { status: 'error', status_message })).reason, SIGN_IN, status_message);
});
// Review round 2: the sign-in words match whole words only, and a prefixed or "is expired" token counts.
test('a message word inside another word is not a sign-in; a prefixed or "is expired" token is', async t => {
for (const status_message of ['assigning a slot', 'logging paused', 'design input'])
assert.equal((await reasonOf(t, { status: 'error', status_message })).reason, NO_REASON, status_message);
for (const status_message of ['access_token expired', 'token is expired', 'refresh-token has been expired', 'please sign in'])
assert.equal((await reasonOf(t, { status: 'error', status_message })).reason, SIGN_IN, status_message);
});
// Review round 3: "relogin", plurals and past forms of "expired" read as a sign-in too.
test('relogin, plural and past forms of an expired sign-in say sign in again', async t => {
for (const status_message of ['relogin required', 'tokens have expired', 'session was expired', 'credentials were expired'])
assert.equal((await reasonOf(t, { status: 'error', status_message })).reason, SIGN_IN, status_message);
});
+3 -1
View File
@@ -8,7 +8,9 @@ import { Broker } from '../src/broker.mjs';
import { Store } from '../src/store.mjs';
const verbs = ['proxy.accounts', 'proxy.login.start', 'proxy.login.submit', 'proxy.login.status',
'proxy.login.cancel', 'proxy.apikey.add', 'proxy.priority.set', 'proxy.group.models.set', 'proxy.keys'];
'proxy.login.cancel', 'proxy.apikey.add', 'proxy.priority.set', 'proxy.group.models.set', 'proxy.keys',
// #1873 account-remove: a tester, agent or user is refused like every other proxy verb.
'proxy.account.remove'];
function setup(t, proxyPage = async () => ({ ok: true })) {
const root = mkdtempSync(join(tmpdir(), 'p4-bus-'));
const store = new Store(root);
+127
View File
@@ -0,0 +1,127 @@
// proxy-ride first slice (Jason 2026-11-21): a riding instance's worker key in the owner
// instance's proxy. Two scratch instances and a stand-in proxy on 127.0.0.1 port 0. The rider's
// system.proxy carries `owner`, as scripts/mosaic-config.mjs gives it. Nothing here reads a real
// config, proxy or secret.
import { test } from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import http from "node:http";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createHash, randomBytes } from "node:crypto";
import { ensureReady, keyPrefix, keyStatus, readKey, rotateKey, sweepKeys } from "../src/triage-key.mjs";
import { ModelKeys } from "../src/model-keys.mjs";
import { loadManagementClient } from "../../model-proxy/src/client.mjs";
const MANAGE = "fxride-manage";
const KEY_PATH = "/v8/management/config/access/api-keys";
const sha = (s) => createHash("sha256").update(s).digest("hex");
const tag = (dir) => sha(dir).slice(0, 12);
const b64 = () => randomBytes(32).toString("base64url");
const dir700 = (p) => { fs.mkdirSync(p, { recursive: true, mode: 0o700 }); fs.chmodSync(p, 0o700); };
async function fixture(t) {
const root = fs.realpathSync(fs.mkdtempSync(join(tmpdir(), "pr-")));
const owner = { configDir: join(root, "oc"), dataRoot: join(root, "od") };
const rider = { configDir: join(root, "rc"), dataRoot: join(root, "rd") };
for (const d of [owner.configDir, join(owner.configDir, "secrets"), owner.dataRoot, join(owner.dataRoot, "proxy"), rider.configDir, join(rider.configDir, "secrets"), rider.dataRoot]) dir700(d);
fs.writeFileSync(join(owner.configDir, "secrets", "proxy-management.key"), MANAGE, { mode: 0o600 });
const O = tag(owner.configDir);
const ownerKeys = { sentinel: `msk-${O}-sentinel-${b64()}`, bus: `msk-${O}-bus-${b64()}`, org: `mso-${O}-demo-key-${b64()}` };
const lockFile = join(owner.dataRoot, "proxy", "mosaic-keys.lock");
const state = { keys: Object.values(ownerKeys), puts: [] };
const server = http.createServer(async (req, res) => {
if (req.url === "/v1/models") {
res.writeHead(state.keys.includes(req.headers.authorization?.slice(7)) ? 200 : 401); res.end("{}"); return;
}
if (req.url !== KEY_PATH || req.headers.authorization !== `Bearer ${MANAGE}`) { res.writeHead(401); res.end("{}"); return; }
if (req.method === "GET") { res.end(JSON.stringify(state.keys)); return; }
const chunks = []; for await (const c of req) chunks.push(c);
state.keys = JSON.parse(Buffer.concat(chunks));
state.puts.push({ locked: fs.existsSync(lockFile) });
res.end('{"status":"ok"}');
});
await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
const port = server.address().port;
t.after(async () => { server.closeAllConnections(); await new Promise((r) => server.close(r)); fs.rmSync(root, { recursive: true, force: true }); });
owner.system = { dataRoot: owner.dataRoot, proxy: { port, binary: "/fixture-unused" } };
rider.system = { dataRoot: rider.dataRoot, proxy: { port, binary: "/fixture-unused", owner: { instance: "owner", configDir: owner.configDir, dataRoot: owner.dataRoot } } };
const opts = (who, extra = {}) => ({ configDir: who.configDir, system: who.system, purpose: "worker", ...extra });
return { root, owner, rider, ownerKeys, state, port, lockFile, opts, receipts: join(owner.dataRoot, "proxy", "mosaic-keys.receipts.jsonl") };
}
test("a rider's worker key lands under the rider's tag, under the owner's lock, and survives the owner's own key work", async (t) => {
const f = await fixture(t);
// The owner writes first, so its lock identity file exists before the rider's write.
await ensureReady(f.opts(f.owner));
const ownerWorker = readKey(f.owner.configDir, { purpose: "worker" }).key;
const putsBefore = f.state.puts.length;
const done = await ensureReady(f.opts(f.rider));
assert.equal(done.action, "issued");
const riderKey = readKey(f.rider.configDir, { purpose: "worker" }).key;
assert.match(riderKey, new RegExp(`^msw-${tag(f.rider.configDir)}-worker-[A-Za-z0-9_-]{43}$`));
assert.ok(riderKey.startsWith(keyPrefix(f.rider.configDir, "worker")));
assert.ok(f.state.keys.includes(riderKey));
assert.ok(f.state.puts.length > putsBefore && f.state.puts.every((p) => p.locked), "every PUT ran under the owner's lock");
assert.equal(fs.existsSync(join(f.rider.dataRoot, "proxy")), false, "the rider has no proxy state of its own");
assert.equal(fs.existsSync(join(f.rider.configDir, "secrets", "proxy-management.key")), false);
const ops = fs.readFileSync(f.receipts, "utf8").trim().split("\n").map((l) => JSON.parse(l).op);
assert.deepEqual(ops, ["worker.key.ensure", "worker.key.ensure"], "the rider's receipt is in the owner's state directory");
for (const k of [...Object.values(f.ownerKeys), ownerWorker]) assert.ok(f.state.keys.includes(k), "the rider removes none of the owner's keys");
// The rider rotates and sweeps its own key; the owner's keys stay.
await rotateKey(f.opts(f.rider));
await sweepKeys(f.opts(f.rider));
for (const k of [...Object.values(f.ownerKeys), ownerWorker]) assert.ok(f.state.keys.includes(k));
const riderNow = readKey(f.rider.configDir, { purpose: "worker" }).key;
// The owner's own worker key work and its bus model key reconcile leave the rider's key.
await rotateKey(f.opts(f.owner));
await sweepKeys(f.opts(f.owner));
await ensureReady(f.opts(f.owner));
const client = await loadManagementClient({ configDir: f.owner.configDir, proxy: f.owner.system.proxy });
const manager = new ModelKeys({ credentials: { status: () => [], modelRecord: () => null }, client, configDir: f.owner.configDir, baseUrl: `http://127.0.0.1:${f.port}/v1` });
const receipt = await manager.reconcile();
assert.ok(receipt.removedKeyIds.length >= 1, "the reconcile did remove the owner's own stale bus key");
assert.ok(f.state.keys.includes(riderNow), "the rider's key survives the owner's reconcile and cleanup");
assert.equal((await ensureReady(f.opts(f.rider))).action, "held");
});
test("the rider's lock identity is the owner's: a writer naming the rider's own config directory refuses", async (t) => {
const f = await fixture(t);
await ensureReady(f.opts(f.owner));
// The same state directory under the rider's own identity is another proxy to the lock.
const wrong = { ...f.rider.system, proxy: { ...f.rider.system.proxy, owner: { ...f.rider.system.proxy.owner, configDir: f.rider.configDir } } };
// Give it every other piece, so only the lock identity differs.
fs.copyFileSync(join(f.owner.configDir, "secrets", "proxy-management.key"), join(f.rider.configDir, "secrets", "proxy-management.key"));
f.state.keys.push(`msk-${tag(f.rider.configDir)}-sentinel-${b64()}`);
const puts = f.state.puts.length;
await assert.rejects(ensureReady({ configDir: f.rider.configDir, system: wrong, purpose: "worker" }), (e) => e.code === "proxy-lock-invalid");
assert.equal(f.state.puts.length, puts, "no write without the owner's identity");
});
test("the owner's fence stops the rider's key write, and the rider's status reports it", async (t) => {
const f = await fixture(t);
await ensureReady(f.opts(f.rider));
const held = readKey(f.rider.configDir, { purpose: "worker" });
fs.writeFileSync(join(f.owner.dataRoot, "proxy", "mosaic-keys.uncertain"), "{}\n", { mode: 0o600 });
const puts = f.state.puts.length;
// A held key goes to the lock while the fence is set, and the lock refuses.
await assert.rejects(ensureReady(f.opts(f.rider)), (e) => e.code === "proxy-uncertain");
await assert.rejects(rotateKey(f.opts(f.rider)), (e) => e.code === "proxy-uncertain");
assert.equal(f.state.puts.length, puts, "no PUT while the owner's fence is set");
assert.deepEqual(readKey(f.rider.configDir, { purpose: "worker" }), held);
const status = await keyStatus(f.opts(f.rider));
assert.equal(status.proxy.fence, true);
assert.equal(status.proxy.state, "read", "status reads the list with the owner's management key and sentinel");
assert.equal(status.proxy.holdsKey, true);
assert.equal(JSON.stringify(status).includes(MANAGE), false);
});
test("a rider without its own secrets directory refuses before any proxy write", async (t) => {
const f = await fixture(t);
fs.rmdirSync(join(f.rider.configDir, "secrets"));
await assert.rejects(ensureReady(f.opts(f.rider)), (e) => e.code === "worker-key-unreadable" && /secrets directory is missing/.test(e.message));
assert.equal(f.state.puts.length, 0);
});
+77
View File
@@ -656,3 +656,80 @@ test('C2-09a: creative bindLaunch without a thread refuses thread-required', t =
assert.equal(ctx.count('SELECT count(*) n FROM threads'), 0);
assert.equal(ctx.count('SELECT count(*) n FROM role_claims'), 0);
});
// #1873 chat-fail F2: the launcher's trusted failureNotice puts one plain line in the conversation
// for each message a refused start or a stalled run left unanswered.
const LIMIT_LINE = "The turn failed: the agent reached a usage limit. Wait for it to reset, or check the limits in Settings.\nDetails: start refused: limit-reached";
const STALL_LINE = "The turn failed: the model service gave no answer in time, so the turn was stopped. Try again, or check the account in Settings > Model setup.\nDetails: stopped: proxy-stall";
const lines = (store, id) => store.all("SELECT * FROM messages WHERE in_reply_to=? AND from_run='bus'", id);
test('#1873 F2: a refused start puts one line per pending message in its session, once, and none for a wait', (t) => {
const { b, store, call, A, B } = twoThreads(t);
b.endLaunch({ business: 'demo', run: 'a-run', reason: 'idle' });
const session = store.get('SELECT session FROM session_defaults WHERE thread=?', A).session;
const notice = (code, extra = {}) => b.failureNotice({ business: 'demo', thread: A, session, code, ...extra });
const one = call('human', 'thread.send', { thread: A, body: 'one' });
const two = call('human', 'thread.send', { thread: A, body: 'two' });
const other = call('human', 'thread.send', { thread: B, body: 'for B' });
// A start that waits for a free place leaves no line.
for (const code of ['capacity-full', 'instance-running', 'legacy-run-live', 'session-workspace-busy', 'processing-generation-changed', 'proxy-lock-busy', 'project-pending', 'processing-halted']) assert.deepEqual(notice(code), { posted: 0 });
assert.deepEqual(notice('limit-reached'), { posted: 2 });
for (const sent of [one, two]) {
const [line] = lines(store, sent.id);
assert.equal(line.body, LIMIT_LINE);
assert.deepEqual([line.class, line.from_role, line.to_role, line.from_run], ['RESULT', 'assistant', 'human', 'bus']);
assert.equal(store.get('SELECT session FROM message_threads WHERE message=?', line.id).session, session, 'the line is in the session');
}
assert.deepEqual(lines(store, other.id), [], "another thread's message gets none");
// Every retry meets the same refusal: still one line each, under any code.
assert.deepEqual(notice('limit-reached'), { posted: 0 });
assert.deepEqual(notice('model-provider-unavailable'), { posted: 0 });
assert.equal(lines(store, one.id).length, 1);
// The message stays pending, so the next start still takes it.
assert.equal(b.threadsState('demo').find((s) => s.thread === A).pending, 2);
const three = call('human', 'thread.send', { thread: A, body: 'three' });
assert.deepEqual(notice('some-new-code'), { posted: 1 });
assert.match(lines(store, three.id)[0].body, /^The turn failed: the assistant couldn't start\. .*\nDetails: start refused: some-new-code$/);
});
test('#1873 F2: a stalled run puts one line on each message delivered to it and unread, thread or role', (t) => {
const { b, store, call, A } = twoThreads(t);
const taken = call('human', 'thread.send', { thread: A, body: 'taken' });
assert.deepEqual(call('a', 'message.receive').map((m) => m.id), [taken.id]);
const waiting = call('human', 'thread.send', { thread: A, body: 'still waiting' });
assert.deepEqual(b.failureNotice({ business: 'demo', run: 'a-run', code: 'proxy-stall' }), { posted: 1 });
assert.equal(lines(store, taken.id)[0].body, STALL_LINE);
assert.deepEqual(lines(store, waiting.id), [], 'a message the run never took waits for the next run');
assert.deepEqual(b.failureNotice({ business: 'demo', run: 'a-run', code: 'proxy-stall' }), { posted: 0 });
// A read message was answered by the turn: no line.
const read = call('human', 'thread.send', { thread: A, body: 'read' });
call('a', 'message.receive');
call('a', 'message.read', { id: read.id });
assert.deepEqual(b.failureNotice({ business: 'demo', run: 'a-run', code: 'proxy-stall' }), { posted: 1 }, 'only waiting, delivered now');
assert.deepEqual(lines(store, read.id), []);
// A role run: the human's message to the chief, outside any thread.
const asked = call('human', 'message.send', { to: 'chief', class: 'REQUEST', body: 'chief, please' });
assert.ok(call('chief', 'message.receive').some((m) => m.id === asked.id));
assert.deepEqual(b.failureNotice({ business: 'demo', run: 'chief-run', code: 'proxy-stall' }), { posted: 1 });
const [line] = lines(store, asked.id);
assert.deepEqual([line.body, line.from_role, line.to_role], [STALL_LINE, 'chief', 'human']);
assert.equal(store.get('SELECT count(*) n FROM message_threads WHERE message=?', line.id).n, 0);
});
test('#1873 F2: failureNotice refuses a malformed record before any write', (t) => {
const { b, store, A } = twoThreads(t);
const session = store.get('SELECT session FROM session_defaults WHERE thread=?', A).session;
const before = store.get('SELECT count(*) n FROM messages').n;
const bad = [
[{ business: 'demo', code: 'limit-reached' }, 'invalid-request'],
[{ business: 'demo', code: 'limit-reached', run: 'a-run', thread: A, session }, 'invalid-request'],
[{ business: 'demo', code: 'Limit Reached', run: 'a-run' }, 'invalid-request'],
[{ business: 'demo', code: 'x'.repeat(65), run: 'a-run' }, 'invalid-request'],
[{ business: 'demo', code: 7, run: 'a-run' }, 'invalid-request'],
[{ business: 'demo', code: 'limit-reached', run: 'a-run', why: 'secret text' }, 'invalid-request'],
[{ business: 'demo', code: 'limit-reached', run: 'nobody-run' }, 'unknown-run'],
[{ business: 'demo', code: 'limit-reached', thread: A, session: 'no-such-session' }, 'session-not-found'],
];
for (const [record, code] of bad) assert.throws(() => b.failureNotice(record), { code }, JSON.stringify(record));
assert.equal(store.get('SELECT count(*) n FROM messages').n, before);
});
+305
View File
@@ -0,0 +1,305 @@
// proxy-scope slice 4 (#1873): the Triage key in the bus. Real organization buses on one instance,
// a stand-in proxy management server on loopback and a temporary config directory; nothing here
// reads a real config, proxy or secret. Each test scans every answer, event, receipt line and
// stderr line for every Triage key the stand-in ever held.
import test from 'node:test';
import assert from 'node:assert/strict';
import fs from 'node:fs';
import http from 'node:http';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { createHash, randomBytes } from 'node:crypto';
import { startBroker } from '../src/runtime.mjs';
import { busOrganization } from '../src/organization-context.mjs';
import { keyPaths, readKey } from '../src/triage-key.mjs';
import { Registry } from '../../seat/src/session.mjs';
import { loadBusiness } from '../../organization/src/organization-config.mjs';
import { resolveInstance } from '../../organization/src/resolve.mjs';
import { businessDoc, rolesCopy, systemFor, writeJson } from '../../organization/tests/helpers.mjs';
const NOW = Date.parse('2026-11-18T12:00:00.000Z');
const MIN = 60_000, HOUR = 60 * MIN, DAY = 24 * HOUR;
const MANAGE = 'fx1873-s4-manage';
const KEY_PATH = '/v8/management/config/access/api-keys';
const sha = s => createHash('sha256').update(s).digest('hex');
const tag = dir => sha(dir).slice(0, 12);
const id = key => sha(key).slice(0, 12);
const b64 = () => randomBytes(32).toString('base64url');
const code = name => e => e.code === name;
// Every Triage key the stand-in ever held, and every text a bus produced, across the file.
const SEEN = new Set();
const TEXTS = [];
function leaks(text) {
for (const key of SEEN) {
const secret = key.slice(-43);
if (text.includes(key) || text.includes(secret)) return key.slice(0, 8);
for (let i = 0; i + 8 <= secret.length; i++) if (text.includes(secret.slice(i, i + 8))) return key.slice(0, 8);
}
return null;
}
// Every stderr line while the test runs: the bus's model-proxy receipt lines go there.
function captureStderr(t) {
const write = process.stderr.write, lines = [];
process.stderr.write = (chunk, ...rest) => { lines.push(String(chunk)); return write.call(process.stderr, chunk, ...rest); };
t.after(() => { process.stderr.write = write; TEXTS.push(lines.join('')); });
return lines;
}
async function instance(t) {
const root = fs.realpathSync(fs.mkdtempSync(join(tmpdir(), 'tkb-')));
const configDir = join(root, 'config');
fs.mkdirSync(join(configDir, 'secrets'), { recursive: true, mode: 0o700 });
fs.chmodSync(configDir, 0o700); fs.chmodSync(join(configDir, 'secrets'), 0o700);
fs.writeFileSync(join(configDir, 'secrets', 'proxy-management.key'), MANAGE, { mode: 0o600 });
fs.mkdirSync(join(root, 'proxy'), { mode: 0o700 });
const sentinel = `msk-${tag(configDir)}-sentinel-${b64()}`, orgKey = `mso-${tag(configDir)}-other-key-${b64()}`;
// triagePuts counts the writes that changed a Triage name; the organization buses' own key
// reconcile writes the list too, and those do not count.
const state = { keys: [sentinel, 'foreign-client-key', orgKey], puts: [], triagePuts: 0 };
const triageNames = list => JSON.stringify(list.filter(k => k.startsWith('mst-')).sort());
const server = http.createServer(async (req, res) => {
if (req.url !== KEY_PATH) {
const key = req.headers.authorization?.slice(7);
res.writeHead(state.keys.includes(key) ? 200 : 401); res.end('{}'); return;
}
if (req.headers.authorization !== `Bearer ${MANAGE}`) { res.writeHead(401); res.end('{}'); return; }
if (req.method === 'GET') { res.end(JSON.stringify(state.keys)); return; }
const chunks = []; for await (const c of req) chunks.push(c);
const next = JSON.parse(Buffer.concat(chunks));
for (const k of next) if (/^mst-.*-[A-Za-z0-9_-]{43}$/.test(k)) SEEN.add(k);
state.puts.push(next);
if (triageNames(next) !== triageNames(state.keys)) state.triagePuts++;
state.keys = next;
res.end('{"status":"ok"}');
});
await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
const port = server.address().port;
const closers = [];
t.after(async () => {
for (const close of closers.reverse()) await close().catch(() => {});
server.closeAllConnections(); await new Promise(resolve => server.close(resolve));
const receipts = join(root, 'proxy', 'mosaic-keys.receipts.jsonl');
if (fs.existsSync(receipts)) TEXTS.push(fs.readFileSync(receipts, 'utf8'));
fs.rmSync(root, { recursive: true, force: true });
});
const rolesDir = rolesCopy(root), system = systemFor(root);
const f = {
root, configDir, port, state, closers, rolesDir, system, sentinel, orgKey, clock: { now: NOW },
prefix: `mst-${tag(configDir)}-triage-`,
file: keyPaths(configDir).file,
record: () => readKey(configDir),
triageKeys: () => f.state.keys.filter(k => k.startsWith(f.prefix)),
receipts: () => {
const p = join(root, 'proxy', 'mosaic-keys.receipts.jsonl');
return fs.existsSync(p) ? fs.readFileSync(p, 'utf8').trim().split('\n').filter(Boolean).map(JSON.parse) : [];
},
// A real organization bus on this instance. `proxy: false` starts it with no proxy settings, as
// a bus on an instance with no proxy.json does.
async bus(orgId, { proxy = true, triageKeys = {} } = {}) {
const doc = businessDoc(root, orgId);
doc.plugins = { ...doc.plugins, harness: ['pi'] };
writeJson(join(configDir, 'businesses', `${orgId}.json`), doc);
const business = loadBusiness(orgId, { dir: configDir, rolesDir }), resolved = {};
for (const name of Object.keys(business.roles)) resolved[name] = resolveInstance({ system, business, instance: name });
const dataRoot = join(root, `${orgId}-bus`);
fs.mkdirSync(dataRoot, { mode: 0o700 }); new Registry(dataRoot);
const bus = await startBroker({ dataRoot, businesses: { [orgId]: busOrganization(business, resolved) },
organization: { dir: configDir, rolesDir, system }, ...(proxy ? { proxy: { port, binary: '/fixture-unused' } } : {}),
org: { id: orgId }, instanceDataRoot: root, instanceConfigDir: configDir,
triageKeys: { intervalMs: 3_600_000, now: () => f.clock.now, ...triageKeys } });
closers.push(() => bus.close());
const cap = bus.broker.bindHuman({ business: orgId, human: business.human ?? doc.human, via: 'cli', outsideAgent: true });
const ask = async (verb, args) => {
try { const answer = await bus.broker.request(cap, { verb, args }); TEXTS.push(JSON.stringify(answer)); return answer; }
catch (error) { TEXTS.push(JSON.stringify({ code: error.code, reason: error.reason, message: error.message, nextStep: error.nextStep })); throw error; }
};
const sweep = async () => { const r = await bus.triageKeySweep(); TEXTS.push(JSON.stringify(r)); return r; };
return { bus, ask, sweep, dataRoot };
},
};
return f;
}
// Every byte under a bus's data root (its store, with events and refusals, and its logs), as text.
function eventsText(b, called = true) {
const out = [];
const walk = dir => {
for (const name of fs.readdirSync(dir)) {
const p = join(dir, name), st = fs.lstatSync(p);
if (st.isDirectory()) walk(p);
else if (st.isFile()) out.push(fs.readFileSync(p).toString('latin1'));
}
};
walk(b.dataRoot);
if (called) assert.ok(out.join('').includes('proxy.triage.key'), 'the store holds the verb\'s events');
return out.join('\n');
}
test('ensure issues one key, a second ensure keeps it, and both answers carry ids only', async t => {
captureStderr(t);
const f = await instance(t);
const a = await f.bus('alpha');
const first = await a.ask('proxy.triage.key', { op: 'ensure' });
assert.equal(first.action, 'issued');
assert.equal(f.triageKeys().length, 1);
const record = f.record();
assert.equal(first.keyId, record.keyId);
assert.equal(first.keyId, id(f.triageKeys()[0]));
assert.deepEqual(first.addedKeyIds, [record.keyId]);
assert.equal(fs.statSync(f.file).mode & 0o777, 0o600);
const second = await a.ask('proxy.triage.key', { op: 'ensure' });
assert.deepEqual(second, { op: 'ensure', action: 'held', addedKeyIds: [], removedKeyIds: [], keyId: record.keyId, rotateBy: record.rotateBy });
assert.equal(f.triageKeys().length, 1, 'ensure twice gives one key');
assert.equal(f.record().key, record.key);
// Through a second riding bus on the same instance: still the one key.
const b = await f.bus('beta');
assert.equal((await b.ask('proxy.triage.key', { op: 'ensure' })).action, 'held');
assert.equal(f.triageKeys().length, 1);
assert.equal(f.state.triagePuts, 1, 'only the first ensure wrote the proxy list');
// The receipt names the change by op and ids.
const receipts = f.receipts().filter(r => r.op?.startsWith('triage.'));
assert.deepEqual(receipts.map(r => [r.op, r.addedKeyIds, r.removedKeyIds]), [['triage.key.ensure', [record.keyId], []]]);
for (const text of [eventsText(a), eventsText(b)]) assert.equal(leaks(text), null);
});
test('ensure replaces a key the proxy lost, and revoke removes every Triage name and the file', async t => {
captureStderr(t);
const f = await instance(t);
const a = await f.bus('alpha');
const issued = await a.ask('proxy.triage.key', { op: 'ensure' });
// The proxy loses the key (removed by hand): ensure issues a new one and names the old as lost.
f.state.keys = f.state.keys.filter(k => !k.startsWith(f.prefix));
const replaced = await a.ask('proxy.triage.key', { op: 'ensure' });
assert.equal(replaced.action, 'replaced');
assert.notEqual(replaced.keyId, issued.keyId);
assert.equal(f.triageKeys().length, 1);
// An orphan in the namespace (a crash left it) goes with revoke too.
const orphan = `${f.prefix}${b64()}`; SEEN.add(orphan);
f.state.keys.push(orphan);
const revoked = await a.ask('proxy.triage.key', { op: 'revoke' });
assert.equal(revoked.action, 'revoked');
assert.deepEqual(revoked.removedKeyIds.sort(), [replaced.keyId, id(orphan)].sort());
assert.equal(f.triageKeys().length, 0);
assert.equal(fs.existsSync(f.file), false);
assert.equal(f.record(), null);
// Revoke again: nothing to do, nothing written.
const puts = f.state.triagePuts;
assert.deepEqual(await a.ask('proxy.triage.key', { op: 'revoke' }), { op: 'revoke', action: 'none', addedKeyIds: [], removedKeyIds: [] });
assert.equal(f.state.triagePuts, puts);
// Namespace guard: the sentinel, the organization key and the foreign key are still there.
for (const k of [f.sentinel, f.orgKey, 'foreign-client-key']) assert.ok(f.state.keys.includes(k), k.slice(0, 4));
assert.equal(leaks(eventsText(a)), null);
});
test('a bus that does not ride the owner\'s proxy refuses triage-no-riding-bus, and its sweep does nothing', async t => {
captureStderr(t);
const f = await instance(t);
const loose = await f.bus('loose', { proxy: false });
await assert.rejects(loose.ask('proxy.triage.key', { op: 'ensure' }), e => e.code === 'triage-no-riding-bus'
&& /^Triage needs one organization that uses the owner's models\./.test(e.nextStep));
await assert.rejects(loose.ask('proxy.triage.key', { op: 'revoke' }), code('triage-no-riding-bus'));
assert.equal(f.state.triagePuts, 0);
// With a key issued through a riding bus and its rotation due, the loose bus still does nothing.
const a = await f.bus('alpha');
await a.ask('proxy.triage.key', { op: 'ensure' });
f.clock.now += 31 * DAY;
assert.deepEqual(await loose.sweep(), { action: 'none' });
assert.equal(f.state.triagePuts, 1);
});
test('the sweep rotates when rotateBy comes, once across two buses that wake together, and again a rotation later', async t => {
captureStderr(t);
const f = await instance(t);
const a = await f.bus('alpha'), b = await f.bus('beta');
const issued = await a.ask('proxy.triage.key', { op: 'ensure' });
// Not due yet: neither bus touches the proxy.
assert.deepEqual([await a.sweep(), await b.sweep()], [{ action: 'none' }, { action: 'none' }]);
assert.equal(f.state.triagePuts, 1);
f.clock.now = Date.parse(`${issued.rotateBy}T00:00:00Z`);
const both = await Promise.all([a.sweep(), b.sweep()]);
assert.deepEqual(both.map(r => r.action).sort(), ['none', 'rotated']);
assert.equal(f.state.triagePuts, 2, 'two buses rotated once');
const rotated = both.find(r => r.action === 'rotated');
const record = f.record();
assert.equal(rotated.keyId, record.keyId);
assert.deepEqual(record.retiring.map(r => r.keyId), [issued.keyId]);
assert.equal(f.triageKeys().length, 2, 'the old key stays live through its grace');
// Inside the grace both buses leave it; at retireAt plus the margin one of them retires it.
const retireAt = Date.parse(record.retiring[0].retireAt);
f.clock.now = retireAt + 10_000;
assert.deepEqual((await Promise.all([a.sweep(), b.sweep()])).map(r => r.action), ['none', 'none']);
f.clock.now = retireAt + 30_000;
const retired = await Promise.all([a.sweep(), b.sweep()]);
assert.deepEqual(retired.map(r => r.action).sort(), ['none', 'retired']);
assert.equal(f.state.triagePuts, 3);
assert.deepEqual(f.triageKeys().map(id), [record.keyId]);
assert.deepEqual(f.record().retiring, []);
// A second rotation 30 days on: again once.
f.clock.now = Date.parse(`${record.rotateBy}T00:00:00Z`) + HOUR;
const again = await Promise.all([b.sweep(), a.sweep()]);
assert.deepEqual(again.map(r => r.action).sort(), ['none', 'rotated']);
assert.equal(f.state.triagePuts, 4);
assert.equal(f.triageKeys().length, 2);
// Namespace guard across every sweep.
for (const k of [f.sentinel, f.orgKey, 'foreign-client-key']) assert.ok(f.state.keys.includes(k), k.slice(0, 4));
const ops = f.receipts().filter(r => r.op?.startsWith('triage.')).map(r => r.op);
assert.deepEqual(ops, ['triage.key.ensure', 'triage.key.rotate', 'triage.key.retire', 'triage.key.rotate']);
for (const bus of [a, b]) assert.equal(leaks(eventsText(bus, bus === a)), null);
});
test('the sweep never issues the first key, and a sweep refusal is a receipt line with a code only', async t => {
const lines = captureStderr(t);
const f = await instance(t);
const a = await f.bus('alpha');
f.clock.now += 400 * DAY;
assert.deepEqual(await a.sweep(), { action: 'none' });
assert.equal(f.state.triagePuts, 0);
assert.equal(f.record(), null);
// A due rotation on a proxy with no sentinel (it may be another proxy): refused, nothing written.
await a.ask('proxy.triage.key', { op: 'ensure' });
f.state.keys = f.state.keys.filter(k => k !== f.sentinel);
f.clock.now += 31 * DAY;
const r = await a.sweep();
assert.deepEqual(r, { action: 'refused', code: 'proxy-not-configured' });
assert.equal(f.state.triagePuts, 1);
const refusal = lines.join('').split('\n').filter(l => l.includes('triage.key.sweep-refused'));
assert.equal(refusal.length, 1);
assert.deepEqual(JSON.parse(refusal[0].replace(/^model-proxy: /, '')), { action: 'triage.key.sweep-refused', code: 'proxy-not-configured', reason: 'triage-key-proxy-open' });
// The same refusal on the next ticks is not logged again within the hour; after it, once more.
const refusals = () => lines.join('').split('\n').filter(l => l.includes('triage.key.sweep-refused')).length;
for (let i = 0; i < 3; i++) { f.clock.now += 30_000; assert.equal((await a.sweep()).code, 'proxy-not-configured'); }
assert.equal(refusals(), 1);
f.clock.now += HOUR;
await a.sweep();
assert.equal(refusals(), 2);
});
test('a revoke stopped between the proxy write and the file is finished by the next revoke; ensure after a stopped revoke replaces the key', async t => {
captureStderr(t);
const f = await instance(t);
let stage = null;
const a = await f.bus('alpha', { triageKeys: { hook: s => { if (s === stage) throw new Error(`stop at ${s}`); } } });
await a.ask('proxy.triage.key', { op: 'ensure' });
stage = 'write';
await assert.rejects(a.ask('proxy.triage.key', { op: 'revoke' }), e => e.code === 'storage-unavailable' && !/stop at/.test(e.message));
assert.equal(f.triageKeys().length, 0, 'the proxy write happened');
assert.ok(f.record(), 'the file stayed');
stage = null;
// ensure now sees a file whose key the proxy lacks: a new key, the old named lost.
const replaced = await a.ask('proxy.triage.key', { op: 'ensure' });
assert.equal(replaced.action, 'replaced');
stage = 'rename';
await assert.rejects(a.ask('proxy.triage.key', { op: 'revoke' }), code('storage-unavailable'));
stage = null;
assert.equal((await a.ask('proxy.triage.key', { op: 'revoke' })).action, 'revoked');
assert.equal(f.record(), null);
assert.equal(f.triageKeys().length, 0);
});
test('no answer, event, receipt line or stderr line in this file holds any Triage key the stand-in saw', () => {
assert.ok(SEEN.size >= 8, `only ${SEEN.size} keys were seen`);
assert.ok(TEXTS.length >= 30, `only ${TEXTS.length} texts`);
for (const text of TEXTS) assert.equal(leaks(text), null);
});
+125
View File
@@ -0,0 +1,125 @@
// proxy-scope slice 4 (#1873): the bus verb proxy.triage.key at the broker. Only the instance
// owner's path-bound proof (a human cap from the cli transport, outside any agent) reaches the
// runtime's hook; an agent, a tester, a Console proof, a grant holder and an organization owner get
// human-required, a reader read-only. The answer and the event carry key ids only, and a hook
// failure that is not a bus refusal becomes storage-unavailable with no text of its own.
import test from 'node:test';
import assert from 'node:assert/strict';
import fs from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { Broker, BusError } from '../src/broker.mjs';
import { Store } from '../src/store.mjs';
import { REFUSALS } from '../src/refusals.mjs';
const role = { authority: { withinRole: ['message.send'], crossRole: [] } };
const code = name => error => error.code === name;
const SUBJECT = { provider: 'fixture', iss: 'https://sso.invalid', sub: 'owner-1' };
// A Triage key as triage-key.mjs makes them, so a leak check has a real-looking value to look for.
const KEY = `mst-0123456789ab-triage-${'Q'.repeat(20)}xY_-${'z'.repeat(19)}`;
function fixture(t, { proxyTriageKey = null } = {}) {
const store = new Store(fs.mkdtempSync(join(tmpdir(), 'tkv-bus-')));
t.after(() => store.close());
const b = new Broker({ store, proxyTriageKey, businesses: { demo: { id: 'demo', human: 'owner', arbiters: { technical: 'chief', delivery: 'chief' }, vars: { 'bus.tester': true }, roles: { chief: { definition: 'chief-of-staff', authority: { withinRole: ['decision.resolve.technical', 'decision.resolve.delivery'], crossRole: [] } }, coder: role } } } });
return {
store, b,
human: b.bindHuman({ business: 'demo', human: 'owner', via: 'cli', outsideAgent: true }),
tester: b.bindTester({ business: 'demo' }), reader: b.bindReader({ business: 'demo' }),
agent: b.bindLaunch({ business: 'demo', role: 'coder', run: 'tkv-run', harness: 'pi' }),
// The owner's Console proof, a granted account that holds org.hire, and an organization owner
// who is not the instance owner: none is the path-bound cli proof.
console: b.bindConsole({ business: 'demo', human: 'owner', subject: SUBJECT }),
grant: b.bindHireConsole({ business: 'demo', actor: 'grantee', subject: { ...SUBJECT, sub: 'grantee-1' }, rights: ['org.hire', 'decide'] }),
orgOwner: b.bindHireConsole({ business: 'demo', actor: 'orgowner', subject: { ...SUBJECT, sub: 'orgowner-1' }, rights: ['org.hire'], owner: true }),
};
}
const events = store => JSON.stringify(store.all('SELECT body FROM events'));
const bodies = store => store.all('SELECT body FROM events').map(r => JSON.parse(r.body));
test('proxy.triage.key refuses an agent, a tester, a Console proof, a grant holder and an organization owner with human-required, and a reader with read-only', async t => {
let calls = 0;
const f = fixture(t, { proxyTriageKey: async () => { calls++; return { action: 'issued', keyId: '0123456789ab', rotateBy: '2026-12-19', addedKeyIds: ['0123456789ab'], removedKeyIds: [] }; } });
for (const op of ['ensure', 'revoke']) {
for (const [name, cap] of [['agent', f.agent], ['tester', f.tester], ['console', f.console], ['grant', f.grant], ['orgOwner', f.orgOwner]])
assert.throws(() => f.b.request(cap, { verb: 'proxy.triage.key', args: { op } }), code('human-required'), `${name} ${op}`);
assert.throws(() => f.b.request(f.reader, { verb: 'proxy.triage.key', args: { op } }), code('read-only'));
}
assert.equal(calls, 0);
// The same verb runs for the owner's cli proof.
const answer = await f.b.request(f.human, { verb: 'proxy.triage.key', args: { op: 'ensure' } });
assert.equal(answer.action, 'issued');
assert.equal(calls, 1);
});
test('proxy.triage.key takes op ensure or revoke and nothing else, and a bus with no hook is not-configured', async t => {
let calls = 0;
const f = fixture(t, { proxyTriageKey: async () => { calls++; return { action: 'none', addedKeyIds: [], removedKeyIds: [] }; } });
for (const args of [{}, { op: 'rotate' }, { op: 'issue' }, { op: 'ensure', key: KEY }, { op: 'revoke', force: true }, { op: ['ensure'] }])
assert.throws(() => f.b.request(f.human, { verb: 'proxy.triage.key', args }), code('invalid-request'), JSON.stringify(args));
assert.throws(() => f.b.request(f.human, { verb: 'proxy.triage.key' }), code('invalid-request'));
assert.equal(calls, 0);
assert.equal(events(f.store).includes(KEY), false);
assert.equal(events(f.store).includes(KEY.slice(-43)), false);
const g = fixture(t);
assert.throws(() => g.b.request(g.human, { verb: 'proxy.triage.key', args: { op: 'ensure' } }), code('not-configured'));
});
test('the answer and the event of each op carry ids only, even when the hook returns more', async t => {
const seen = [];
const f = fixture(t, { proxyTriageKey: async op => {
seen.push(op);
// A hook that returns the key beside the ids: the broker copies the named fields only.
return op === 'ensure'
? { action: 'issued', keyId: '0123456789ab', rotateBy: '2026-12-19', addedKeyIds: ['0123456789ab'], removedKeyIds: ['ba9876543210'], key: KEY, old: { key: KEY } }
: { action: 'revoked', addedKeyIds: [], removedKeyIds: ['0123456789ab'], key: KEY };
} });
const ensure = await f.b.request(f.human, { verb: 'proxy.triage.key', args: { op: 'ensure' } });
assert.deepEqual(ensure, { op: 'ensure', action: 'issued', addedKeyIds: ['0123456789ab'], removedKeyIds: ['ba9876543210'], keyId: '0123456789ab', rotateBy: '2026-12-19' });
const revoke = await f.b.request(f.human, { verb: 'proxy.triage.key', args: { op: 'revoke' } });
assert.deepEqual(revoke, { op: 'revoke', action: 'revoked', addedKeyIds: [], removedKeyIds: ['0123456789ab'] });
assert.deepEqual(seen, ['ensure', 'revoke']);
assert.deepEqual(bodies(f.store).filter(b => b.call === 'proxy.triage.key'), [{ call: 'proxy.triage.key', ...ensure }, { call: 'proxy.triage.key', ...revoke }]);
for (const text of [JSON.stringify(ensure), JSON.stringify(revoke), events(f.store)]) {
assert.equal(text.includes(KEY), false);
assert.equal(text.includes(KEY.slice(-43)), false);
}
});
test('an answer the broker cannot vouch for refuses proxy-client-response and records no allowed event', async t => {
const answers = [
null, { action: 'issued' },
{ action: 'deleted', keyId: '0123456789ab', addedKeyIds: [], removedKeyIds: [] },
{ action: 'issued', keyId: KEY, addedKeyIds: [], removedKeyIds: [] },
{ action: 'issued', keyId: '0123456789ab', addedKeyIds: [KEY], removedKeyIds: [] },
{ action: 'held', keyId: '0123456789ab', addedKeyIds: [], removedKeyIds: 'none' },
];
let next = 0;
const f = fixture(t, { proxyTriageKey: async () => answers[next++] });
for (const answer of answers)
await assert.rejects(f.b.request(f.human, { verb: 'proxy.triage.key', args: { op: 'ensure' } }), code('proxy-client-response'), JSON.stringify(answer));
await assert.rejects(Promise.resolve().then(() => fixture(t, { proxyTriageKey: async () => ({ action: 'issued', addedKeyIds: [], removedKeyIds: [] }) }))
.then(g => g.b.request(g.human, { verb: 'proxy.triage.key', args: { op: 'revoke' } })), code('proxy-client-response'));
assert.equal(bodies(f.store).some(b => b.call === 'proxy.triage.key'), false);
assert.equal(events(f.store).includes(KEY.slice(-43)), false);
});
test('a bus refusal from the hook passes through with its guidance; any other failure is storage-unavailable and names nothing from the error', async t => {
const f = fixture(t, { proxyTriageKey: async op => { throw op === 'ensure' ? new BusError('triage-no-riding-bus') : Object.assign(new Error(`boom ${KEY} /home/x`), { code: 'EACCES' }); } });
await assert.rejects(f.b.request(f.human, { verb: 'proxy.triage.key', args: { op: 'ensure' } }), e => e.code === 'triage-no-riding-bus'
&& /Triage needs one organization that uses the owner's models/.test(e.nextStep ?? REFUSALS['triage-no-riding-bus'].nextStep));
await assert.rejects(f.b.request(f.human, { verb: 'proxy.triage.key', args: { op: 'revoke' } }), e => e.code === 'storage-unavailable'
&& !JSON.stringify({ ...e, message: e.message }).includes(KEY.slice(-43)) && !String(e.message).includes('/home/x'));
assert.equal(events(f.store).includes(KEY.slice(-43)), false);
});
test('the two slice 4 codes have refusal-table entries with a next step and no absolute path', () => {
for (const c of ['triage-no-riding-bus', 'triage-key-rotated']) {
const entry = REFUSALS[c];
assert.ok(entry, c);
assert.equal(typeof entry.nextStep, 'string', c);
assert.ok(entry.nextStep.length > 20, c);
assert.equal(/\/(home|mnt|srv|tmp)\//.test(entry.nextStep), false, c);
}
assert.match(REFUSALS['triage-no-riding-bus'].nextStep, /^Triage needs one organization that uses the owner's models\./);
});
+579
View File
@@ -0,0 +1,579 @@
// worker-key-purpose (S3 of reports/worker-proxy-simple.md): triage-key.mjs with purpose "worker".
// The worker key `msw-<tag>-worker-<43>` in <configDir>/secrets/worker/model, its mirrored
// worker-key-* refusals, the lock-free launch path ensureReady, the post-write probe, the namespace
// fence in both directions and the retire guard. A stand-in proxy on 127.0.0.1 port 0 and scratch
// config directories only; nothing here reads a real config, proxy or secret.
import { test } from "node:test";
import assert from "node:assert/strict";
import { spawn } from "node:child_process";
import fs from "node:fs";
import http from "node:http";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createHash, randomBytes } from "node:crypto";
import {
PROBE_TIMING, PURPOSES, ensureKey, ensureReady, issueKey, keyPaths, keyPrefix, keyStatus, readKey, retireHeld, retireKeys,
revokeKeys, rotateKey, runtimeLive, runtimeStartMs, sweepKeys, RETIRE_LAUNCH_MARGIN_MS,
} from "../src/triage-key.mjs";
import { processIdentity } from "../../processing-control/src/runtimes.mjs";
import { ISSUE_PROBE_TIMEOUT_MS, ISSUE_REPROBE_MIN_MS, REVOKE_POLL_MS } from "../src/model-keys.mjs";
const NOW = Date.parse("2026-11-21T12:00:00.000Z");
const MIN = 60_000, HOUR = 60 * MIN, DAY = 24 * HOUR;
const MANAGE = "fxwk-manage";
const KEY_PATH = "/v8/management/config/access/api-keys";
const sha = (s) => createHash("sha256").update(s).digest("hex");
const tag = (dir) => sha(dir).slice(0, 12);
const id = (key) => sha(key).slice(0, 12);
const b64 = () => randomBytes(32).toString("base64url");
const iso = (ms) => new Date(ms).toISOString();
const MODULE = new URL("../src/triage-key.mjs", import.meta.url).href;
// Every msw-/mst- key the stand-in ever saw, and every text a call produced, for the leak scans.
const SEEN = new Set();
const OUTPUTS = [];
function leaks(text, keys = SEEN) {
for (const key of keys) {
const secret = key.slice(-43);
if (text.includes(key) || text.includes(secret)) return true;
for (let i = 0; i + 8 <= secret.length; i++) if (text.includes(secret.slice(i, i + 8))) return true;
}
return false;
}
// A stand-in proxy. A PUT records whether the instance proxy lock file existed while it ran.
async function fixture(t) {
const root = fs.realpathSync(fs.mkdtempSync(join(tmpdir(), "wk-")));
const configDir = join(root, "config");
fs.mkdirSync(join(configDir, "secrets"), { recursive: true, mode: 0o700 });
fs.chmodSync(configDir, 0o700); fs.chmodSync(join(configDir, "secrets"), 0o700);
fs.writeFileSync(join(configDir, "secrets", "proxy-management.key"), MANAGE, { mode: 0o600 });
fs.mkdirSync(join(root, "proxy"), { mode: 0o700 });
const T = tag(configDir);
const others = {
sentinel: `msk-${T}-sentinel-${b64()}`, bus: `msk-${T}-bus-${b64()}`, org: `mso-${T}-demo-key-${b64()}`,
foreign: "foreign-client-key", otherWorker: `msw-${"f".repeat(12)}-worker-${b64()}`,
};
const lockFile = join(root, "proxy", "mosaic-keys.lock");
const state = { keys: Object.values(others), gets: 0, puts: [], probes: [], refuse401: 0, ignorePut: false, onGet: null };
const server = http.createServer(async (req, res) => {
if (req.url === "/v1/models") {
const key = req.headers.authorization?.slice(7);
state.probes.push(key);
if (state.dropProbe) { req.socket.destroy(); return; }
if (state.refuse401 > 0) { state.refuse401--; res.writeHead(401); res.end("{}"); return; }
res.writeHead(state.keys.includes(key) ? 200 : 401); res.end("{}"); return;
}
if (req.url !== KEY_PATH || req.headers.authorization !== `Bearer ${MANAGE}`) { res.writeHead(401); res.end("{}"); return; }
if (req.method === "GET") { state.gets++; const body = JSON.stringify(state.keys); state.onGet?.(state.gets); res.end(body); return; }
const chunks = []; for await (const c of req) chunks.push(c);
const next = JSON.parse(Buffer.concat(chunks));
for (const k of next) if (/^ms[tw]-.*-[A-Za-z0-9_-]{43}$/.test(k)) SEEN.add(k);
state.puts.push({ keys: next, locked: fs.existsSync(lockFile) });
if (!state.ignorePut) state.keys = next;
res.end('{"status":"ok"}');
});
await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
const port = server.address().port;
t.after(async () => {
if (fs.existsSync(join(root, "proxy", "mosaic-keys.receipts.jsonl"))) OUTPUTS.push(fs.readFileSync(join(root, "proxy", "mosaic-keys.receipts.jsonl"), "utf8"));
server.closeAllConnections(); await new Promise((resolve) => server.close(resolve));
fs.rmSync(root, { recursive: true, force: true });
});
const f = {
root, configDir, state, others, port, lockFile, now: NOW,
system: { dataRoot: root, proxy: { port, binary: "/fixture-unused" } },
worker: keyPaths(configDir, "worker"), triage: keyPaths(configDir, "triage"),
receipts: join(root, "proxy", "mosaic-keys.receipts.jsonl"),
opts(extra = {}) {
const stages = [];
const o = { configDir, system: f.system, now: () => f.now, purpose: "worker", hook: (s) => stages.push(s), ...extra };
o.stages = stages;
return o;
},
mine: (purpose = "worker") => state.keys.filter((k) => k.startsWith(keyPrefix(configDir, purpose))),
read: (purpose = "worker") => readKey(configDir, { purpose }),
// Every file under the root with its size and mtime, for "wrote nothing".
snapshot() {
const out = {};
const walk = (dir) => { for (const name of fs.readdirSync(dir)) { const p = join(dir, name), st = fs.lstatSync(p); out[p] = st.isDirectory() ? "dir" : `${st.size}:${st.mtimeMs}`; if (st.isDirectory()) walk(p); } };
walk(root);
return out;
},
// Writes a worker record by hand: the current key, plus retiring entries.
seed({ issuedAt = NOW - DAY, rotateBy, retiring = [], inList = true } = {}) {
const key = keyPrefix(configDir, "worker") + b64();
SEEN.add(key);
const record = { version: 1, key, keyId: id(key), issuedAt: iso(issuedAt), rotateBy: rotateBy ?? iso(issuedAt + 30 * DAY).slice(0, 10), retiring };
fs.mkdirSync(f.worker.dir, { recursive: true, mode: 0o700 });
fs.writeFileSync(f.worker.file, JSON.stringify(record), { mode: 0o600 });
if (inList) state.keys.push(key);
for (const r of retiring) if (r.inList !== false && !state.keys.includes(r.key)) state.keys.push(r.key);
return record;
},
retiring(retireAt, { inList = true } = {}) {
const key = keyPrefix(configDir, "worker") + b64();
SEEN.add(key);
return { key, keyId: id(key), issuedAt: iso(NOW - 40 * DAY), retireAt: iso(retireAt), ...(inList ? {} : { inList }) };
},
};
return f;
}
// readKey refuses unknown fields, so the seed drops the test-only inList marker.
const clean = (r) => { const { inList, ...rest } = r; return rest; };
async function refused(promise, code) {
try { await promise; } catch (e) { OUTPUTS.push(e.message); assert.equal(e.code, code, e.message); return e; }
assert.fail(`expected ${code}`);
}
const deadRuntime = () => ({ phase: "running", supervisor: { pid: 999_999_999, startTime: "1", boot: "00000000-0000-4000-8000-000000000000" } });
test("the worker purpose names its own prefix, file and codes, and the Triage purpose is unchanged", async (t) => {
const f = await fixture(t);
assert.equal(keyPrefix(f.configDir), `mst-${tag(f.configDir)}-triage-`);
assert.equal(keyPrefix(f.configDir, "worker"), `msw-${tag(f.configDir)}-worker-`);
assert.equal(f.triage.file, join(f.configDir, "secrets", "triage", "model"));
assert.equal(f.worker.file, join(f.configDir, "secrets", "worker", "model"));
assert.deepEqual(Object.keys(PURPOSES), ["triage", "worker"]);
assert.throws(() => keyPrefix(f.configDir, "org"), TypeError);
await assert.rejects(ensureReady(f.opts({ purpose: "bus" })), TypeError);
const done = await ensureReady(f.opts());
assert.equal(done.action, "issued");
const record = f.read();
assert.match(record.key, new RegExp(`^msw-${tag(f.configDir)}-worker-[A-Za-z0-9_-]{43}$`));
assert.equal((fs.statSync(f.worker.file).mode & 0o777), 0o600);
assert.equal((fs.statSync(f.worker.dir).mode & 0o777), 0o700);
assert.equal(fs.existsSync(f.triage.dir), false, "a worker call leaves the Triage directory alone");
assert.equal(readKey(f.configDir), null, "the Triage reader does not see the worker key");
assert.deepEqual(f.mine(), [record.key]);
for (const k of Object.values(f.others)) assert.ok(f.state.keys.includes(k));
const lines = fs.readFileSync(f.receipts, "utf8").trim().split("\n").map((l) => JSON.parse(l));
assert.deepEqual(lines.map((l) => l.op), ["worker.key.ensure"]);
assert.deepEqual(lines[0].addedKeyIds, [record.keyId]);
assert.ok(f.state.puts.every((p) => p.locked), "every PUT ran under the lock");
assert.equal(f.state.probes.at(-1), record.key, "the key was probed after the write");
});
test("each worker-key-* refusal mirrors its triage-key-* code", async (t) => {
const f = await fixture(t);
const noProxy = { configDir: f.configDir, system: { dataRoot: f.root } };
await refused(ensureReady({ ...noProxy, purpose: "worker" }), "worker-key-no-proxy");
await refused(ensureKey({ ...noProxy }), "triage-key-no-proxy");
await refused(rotateKey(f.opts()), "worker-key-not-issued");
await issueKey(f.opts());
const issued = await refused(issueKey(f.opts()), "worker-key-issued");
assert.equal(issued.message, "a worker key is already issued");
fs.chmodSync(f.worker.file, 0o644);
await refused(ensureReady(f.opts()), "worker-key-unreadable");
fs.chmodSync(f.worker.file, 0o600);
const sentinel = f.others.sentinel;
f.state.keys = f.state.keys.filter((k) => k !== sentinel);
await refused(ensureReady(f.opts()), "worker-key-proxy-open");
f.state.keys.push(sentinel);
f.state.onGet = (n) => { if (n % 2 === 0) f.state.keys.push(`unrelated-${n}`); };
const raced = await refused(rotateKey(f.opts()), "worker-key-raced");
assert.match(raced.message, /run the launch again/);
assert.doesNotMatch(raced.message, /verb/);
f.state.onGet = null;
f.state.ignorePut = true;
await refused(rotateKey(f.opts()), "worker-key-unconfirmed");
f.state.ignorePut = false;
fs.rmSync(join(f.root, "proxy", "mosaic-keys.uncertain"), { force: true });
fs.chmodSync(f.receipts, 0o644);
await refused(rotateKey(f.opts()), "worker-key-receipts-unsafe");
fs.chmodSync(f.receipts, 0o600);
const unwritten = await refused(rotateKey(f.opts({ hook: (s) => { if (s === "rename") throw new Error("stop"); } })), "worker-key-unwritten");
assert.match(unwritten.message, /run the launch again/);
assert.doesNotMatch(unwritten.message, /triage/i);
const unsynced = await refused(rotateKey(f.opts({ hook: (s) => { if (s === "sync") throw new Error("stop"); } })), "worker-key-unsynced");
assert.match(unsynced.message, /secrets\/worker\/model/);
});
test("ensureReady on a held worker key takes no lock, writes nothing and probes the key", async (t) => {
const f = await fixture(t);
await ensureReady(f.opts());
const record = f.read();
// The lock's identity file is a write too; the fast path must not make it.
fs.rmSync(join(f.root, "proxy", "mosaic-proxy.identity"));
const before = f.snapshot(), puts = f.state.puts.length, probes = f.state.probes.length;
const o = f.opts();
const done = await ensureReady(o);
assert.deepEqual(o.stages, ["fast"]);
assert.deepEqual(done, { action: "held", keyId: record.keyId, rotateBy: record.rotateBy, addedKeyIds: [], removedKeyIds: [] });
assert.deepEqual(f.snapshot(), before, "no file changed or appeared");
assert.equal(f.state.puts.length, puts);
assert.deepEqual(f.state.probes.slice(probes), [record.key]);
// A Triage key takes the same fast path.
await ensureReady(f.opts({ purpose: "triage" }));
const triageRecord = f.read("triage");
fs.rmSync(join(f.root, "proxy", "mosaic-proxy.identity"));
const o2 = f.opts({ purpose: "triage" });
const snap = f.snapshot();
assert.equal((await ensureReady(o2)).action, "held");
assert.deepEqual(o2.stages, ["fast"]);
assert.deepEqual(f.snapshot(), snap);
assert.equal(f.state.probes.at(-1), triageRecord.key);
});
test("ensureReady takes the lock for every state that needs a write, and only writes under it", async (t) => {
const cases = [
["no file", (f) => {}, "issued"],
["key gone from the list", (f) => { const r = f.seed({ inList: false }); return r; }, "replaced"],
["an orphan in the worker namespace", (f) => { f.seed(); f.state.keys.push(keyPrefix(f.configDir, "worker") + b64()); }, "held"],
["a malformed name in the worker namespace", (f) => { f.seed(); f.state.keys.push(keyPrefix(f.configDir, "worker") + "short"); }, "held"],
["a retiring key the proxy lost", (f) => { const r = f.retiring(NOW + HOUR, { inList: false }); f.seed({ retiring: [clean(r)] }); f.state.keys = f.state.keys.filter((k) => k !== r.key); }, "held"],
["a leftover temp file", (f) => { f.seed(); fs.writeFileSync(join(f.worker.dir, `.model.1.${"a".repeat(12)}.tmp`), "x", { mode: 0o600 }); }, "held"],
["rotateBy has come", (f) => { f.seed({ issuedAt: NOW - 31 * DAY }); }, "rotated"],
["a retiring key past its grace, no live runs", (f) => { const r = f.retiring(NOW - MIN); f.seed({ retiring: [clean(r)] }); }, "retired"],
];
for (const [name, setup, action] of cases) {
await t.test(name, async (t) => {
const f = await fixture(t);
setup(f);
const o = f.opts({ runtimes: () => [] });
const done = await ensureReady(o);
assert.equal(o.stages[0], "locked", name);
assert.equal(done.action, action);
assert.ok(f.state.puts.every((p) => p.locked), "no PUT ran without the lock");
assert.deepEqual(fs.readdirSync(f.worker.dir), ["model"], "no temp file is left");
const record = f.read();
assert.deepEqual([...f.mine()].sort(), [record.key, ...record.retiring.map((r) => r.key)].sort(), "the list holds exactly the file's keys");
// The next launch finds nothing to write.
const o2 = f.opts({ runtimes: () => [] });
await ensureReady(o2);
assert.deepEqual(o2.stages, ["fast"], `${name}: settled`);
});
}
await t.test("a set fence sends the launch to the lock, which refuses", async (t) => {
const f = await fixture(t);
f.seed();
fs.writeFileSync(join(f.root, "proxy", "mosaic-keys.uncertain"), "{}", { mode: 0o600 });
const o = f.opts();
await refused(ensureReady(o), "proxy-uncertain");
assert.deepEqual(o.stages, ["locked"]);
assert.equal(f.state.puts.length, 0);
});
});
test("ensureReady probes a just-written key again while the proxy answers 401, then refuses", async (t) => {
assert.deepEqual({ ...PROBE_TIMING }, { timeoutMs: ISSUE_PROBE_TIMEOUT_MS, reprobeMinMs: ISSUE_REPROBE_MIN_MS, pollMs: REVOKE_POLL_MS }, "the bus's probe timing");
const f = await fixture(t);
f.state.refuse401 = 3;
const done = await ensureReady(f.opts());
assert.equal(done.action, "issued");
assert.equal(f.state.probes.length, 4, "three 401s, then the 200");
f.state.refuse401 = 1e9;
const started = Date.now();
await refused(ensureReady(f.opts()), "proxy-key-refused");
assert.ok(Date.now() - started >= 1500, "the 401 was probed until the deadline");
f.state.refuse401 = 0;
// A proxy that drops the model route gives proxy-down, on the fast path and after a write.
f.state.dropProbe = true;
await refused(ensureReady(f.opts()), "proxy-down");
fs.rmSync(f.worker.file);
await refused(ensureReady(f.opts()), "proxy-down");
f.state.dropProbe = false;
assert.equal((await ensureReady(f.opts())).action, "held", "the key written before the probe failed is kept");
});
test("a fast-path probe the proxy refuses goes to the lock once, which re-adds the key", async (t) => {
const f = await fixture(t);
const record = f.seed();
// A list writer outside the lock drops the key after this launch read the list.
f.state.onGet = (n) => { if (n === 1) f.state.keys = f.state.keys.filter((k) => k !== record.key); };
const o = f.opts();
const done = await ensureReady(o);
f.state.onGet = null;
assert.deepEqual(o.stages.filter((s) => s === "fast" || s === "locked"), ["fast", "locked"]);
assert.equal(done.action, "replaced");
assert.ok(f.state.puts.length > 0 && f.state.puts.every((p) => p.locked), "the re-add ran under the lock");
assert.equal(f.read().keyId, done.keyId);
assert.ok(f.state.keys.includes(f.read().key));
// A key the list holds but the proxy still refuses: the locked pass changes nothing, then refuses.
f.state.refuse401 = 1e9;
const puts = f.state.puts.length;
const o2 = f.opts();
await refused(ensureReady(o2), "proxy-key-refused");
f.state.refuse401 = 0;
assert.deepEqual(o2.stages.filter((s) => s === "fast" || s === "locked"), ["fast", "locked"]);
assert.equal(f.state.puts.length, puts, "nothing was written");
});
test("change() refuses a plan that would remove a key outside the purpose's namespace", async (t) => {
for (const purpose of ["worker", "triage"]) {
await t.test(purpose, async (t) => {
const f = await fixture(t);
await issueKey(f.opts({ purpose }));
const puts = f.state.puts.length;
const other = purpose === "worker" ? f.others.otherWorker : f.others.sentinel;
const planSeam = (result, before) => ({ ...result, after: (result.after ?? before).filter((k) => k !== other) });
const e = await refused(rotateKey(f.opts({ purpose, planSeam })), `${purpose}-key-failed`);
assert.match(e.message, /outside the (worker|Triage) namespace/);
assert.equal(f.state.puts.length, puts, "no PUT ran");
assert.ok(f.state.keys.includes(other));
});
}
});
test("two parallel worker ensureReady calls both succeed and leave exactly one key", async (t) => {
const f = await fixture(t);
const results = await Promise.all([ensureReady(f.opts()), ensureReady(f.opts())]);
assert.deepEqual(results.map((r) => r.action).sort(), ["held", "issued"]);
const record = f.read();
assert.deepEqual(f.mine(), [record.key]);
assert.equal(new Set(results.map((r) => r.keyId)).size, 1);
assert.ok(f.state.puts.length === 1 && f.state.puts[0].locked);
// The same from two processes.
fs.rmSync(f.worker.file);
f.state.keys = f.state.keys.filter((k) => !k.startsWith(keyPrefix(f.configDir, "worker")));
const script = `
import { ensureReady } from ${JSON.stringify(MODULE)};
const p = JSON.parse(process.env.WK_OPTS);
const done = await ensureReady({ ...p, purpose: "worker" });
process.stdout.write(JSON.stringify(done));`;
const child = () => new Promise((resolve) => {
const c = spawn(process.execPath, ["--input-type=module", "-e", script], { env: { PATH: process.env.PATH, HOME: process.env.HOME, TMPDIR: process.env.TMPDIR, WK_OPTS: JSON.stringify({ configDir: f.configDir, system: f.system }) }, stdio: ["ignore", "pipe", "pipe"] });
let out = "", err = "";
c.stdout.on("data", (d) => { out += d; }); c.stderr.on("data", (d) => { err += d; });
c.on("close", (code) => resolve({ code, out, err }));
});
const kids = await Promise.all([child(), child(), child()]);
for (const k of kids) { OUTPUTS.push(k.out, k.err); assert.equal(k.code, 0, k.err); }
const actions = kids.map((k) => JSON.parse(k.out).action).sort();
assert.deepEqual(actions, ["held", "held", "issued"]);
assert.deepEqual(f.mine(), [f.read().key]);
assert.ok(f.state.puts.every((p) => p.locked));
});
test("a Triage call never removes a worker key, and a worker call never removes a Triage, org, bus or foreign key", async (t) => {
await t.test("Triage calls leave msw- names alone", async (t) => {
const f = await fixture(t);
const workerKey = keyPrefix(f.configDir, "worker") + b64();
const malformed = keyPrefix(f.configDir, "worker") + "x";
f.state.keys.push(workerKey, malformed);
const keep = [workerKey, malformed, ...Object.values(f.others)];
const triage = (extra = {}) => ({ configDir: f.configDir, system: f.system, now: () => f.now, ...extra });
const steps = [
() => issueKey(triage()), () => ensureKey(triage()), () => ensureReady(triage()), () => rotateKey(triage()),
() => { f.now = NOW + DAY; return retireKeys(triage()); }, () => { f.now = NOW + 40 * DAY; return sweepKeys(triage()); },
() => revokeKeys(triage()),
];
for (const step of steps) {
await step();
for (const k of keep) assert.ok(f.state.keys.includes(k), "a non-Triage key stayed");
}
assert.deepEqual(f.mine("triage"), []);
assert.equal(fs.existsSync(f.worker.file), false);
});
await t.test("worker calls leave mst-, mso-, msk- and foreign names alone", async (t) => {
const f = await fixture(t);
const triageKey = keyPrefix(f.configDir, "triage") + b64();
const triageMalformed = keyPrefix(f.configDir, "triage") + "x";
f.state.keys.push(triageKey, triageMalformed);
const keep = [triageKey, triageMalformed, ...Object.values(f.others)];
const steps = [
() => issueKey(f.opts()), () => ensureKey(f.opts()), () => ensureReady(f.opts()), () => rotateKey(f.opts()),
() => { f.now = NOW + DAY; return retireKeys(f.opts({ runtimes: () => [] })); },
() => { f.now = NOW + 40 * DAY; return ensureReady(f.opts({ runtimes: () => [] })); },
() => { f.now = NOW + 80 * DAY; return sweepKeys(f.opts({ runtimes: () => [] })); },
() => revokeKeys(f.opts()),
];
for (const step of steps) {
await step();
for (const k of keep) assert.ok(f.state.keys.includes(k), "a non-worker key stayed");
}
assert.deepEqual(f.mine(), []);
assert.equal(fs.existsSync(f.worker.file), false, "revoke removed the worker file");
assert.equal(fs.existsSync(f.triage.dir), false, "no Triage file was made");
// The other instance's worker key (another tag) is another namespace too.
assert.ok(f.state.keys.includes(f.others.otherWorker));
});
});
test("retireHeld keeps an old worker key on every doubt and lets it go only when no older run is live", () => {
const rotatedAtMs = NOW;
const run = (started, extra = {}) => ({ phase: "running", started, ...extra });
const seams = { live: (r) => r.live ?? true, startedAt: (r) => r.started };
assert.equal(retireHeld({ rotatedAtMs, ...seams }), true, "no runtimes source");
assert.equal(retireHeld({ runtimes: () => { throw new Error("x"); }, rotatedAtMs, ...seams }), true, "the source throws");
assert.equal(retireHeld({ runtimes: () => ({}), rotatedAtMs, ...seams }), true, "not a list");
assert.equal(retireHeld({ runtimes: () => [], rotatedAtMs: NaN, ...seams }), true, "no rotation time");
assert.equal(retireHeld({ runtimes: () => [], rotatedAtMs, ...seams }), false, "no runs");
assert.equal(retireHeld({ runtimes: () => [run(NOW - HOUR)], rotatedAtMs, ...seams }), true, "a run from before the rotation");
assert.equal(retireHeld({ runtimes: () => [run(NOW + RETIRE_LAUNCH_MARGIN_MS)], rotatedAtMs, ...seams }), true, "a run inside the launch margin");
assert.equal(retireHeld({ runtimes: () => [run(NOW + RETIRE_LAUNCH_MARGIN_MS + 1)], rotatedAtMs, ...seams }), false, "a run after the margin");
assert.equal(RETIRE_LAUNCH_MARGIN_MS, 10 * MIN);
assert.equal(retireHeld({ runtimes: () => [run(NOW + 5 * MIN)], rotatedAtMs, ...seams }), true, "a run five minutes after the rotation may hold the old key");
assert.equal(retireHeld({ runtimes: () => [run(NOW + 5 * MIN)], rotatedAtMs, marginMs: MIN, ...seams }), false, "marginMs is a seam");
assert.equal(retireHeld({ runtimes: () => [run(null)], rotatedAtMs, ...seams }), true, "a run whose start is unknown");
assert.equal(retireHeld({ runtimes: () => [run(NOW - HOUR, { phase: "ended" })], rotatedAtMs, ...seams }), false, "an ended run");
assert.equal(retireHeld({ runtimes: () => [run(NOW - HOUR, { live: false })], rotatedAtMs, ...seams }), false, "a run proved dead");
assert.equal(retireHeld({ runtimes: () => [run(NOW - HOUR, { live: null })], rotatedAtMs, ...seams }), true, "liveness unknown");
assert.equal(retireHeld({ runtimes: () => [run(NOW - HOUR)], rotatedAtMs, live: () => { throw new Error("x"); }, startedAt: () => 0 }), true, "a seam throws");
assert.equal(retireHeld({ runtimes: () => [run(NOW + 2 * RETIRE_LAUNCH_MARGIN_MS), run(NOW - HOUR)], rotatedAtMs, ...seams }), true, "one older run is enough");
});
test("runtimeLive and runtimeStartMs read the supervisor from /proc", async () => {
const me = processIdentity(process.pid);
const record = { phase: "running", supervisor: me };
assert.equal(runtimeLive(record), true);
const expected = Date.now() - process.uptime() * 1000;
assert.ok(Math.abs(runtimeStartMs(record) - expected) < 3000, "start ticks at 100 a second plus btime match this process's start");
assert.equal(runtimeLive({ ...record, supervisor: { ...me, startTime: String(Number(me.startTime) + 1) } }), false, "another process on the same pid");
assert.equal(runtimeStartMs({ ...record, supervisor: { ...me, startTime: "1" } }), null);
assert.equal(runtimeLive({ phase: "running" }), null, "no supervisor: unknown");
const child = spawn(process.execPath, ["-e", "setTimeout(() => {}, 60000)"], { stdio: "ignore" });
await new Promise((resolve) => setTimeout(resolve, 100));
const kid = processIdentity(child.pid);
assert.equal(runtimeLive({ phase: "running", supervisor: kid }), true);
child.kill("SIGKILL");
await new Promise((resolve) => child.on("exit", resolve));
assert.equal(runtimeLive({ phase: "running", supervisor: kid }), false, "a dead supervisor with no pid of its own");
assert.equal(runtimeLive({ phase: "running", supervisor: kid, pid: process.pid, startTime: me.startTime, boot: me.boot }), true, "the runtime's own pid lives on");
assert.equal(runtimeLive(deadRuntime()), false);
// A container runtime has no pid, and its dead docker client is no proof the container exited.
const container = { phase: "running", kind: "container", containerName: `mosaic-${"a".repeat(12)}-${"0".repeat(8)}-0000-4000-8000-${"0".repeat(12)}`, supervisor: kid };
assert.equal(runtimeLive(container), null, "a container whose launcher died is unknown, not gone");
assert.equal(runtimeLive({ ...container, containerName: undefined }), null, "kind container alone is enough");
assert.equal(runtimeLive({ ...container, kind: "task" }), null, "a task record with a container name, the shape a real worker run registers");
assert.equal(retireHeld({ runtimes: () => [container], rotatedAtMs: NOW }), true, "so it holds the old key");
assert.equal(retireHeld({ runtimes: () => [{ ...container, phase: "ended" }], rotatedAtMs: NOW }), false, "until its record is ended");
assert.equal(runtimeLive({ ...container, supervisor: me }), true, "a live launcher is live");
// A pid this process may not signal or read is unknown, never dead (processing-control's own check).
assert.equal(runtimeLive({ phase: "running", supervisor: { ...me, boot: "not-a-boot-id" } }), null, "an identity the check cannot use");
});
test("a worker key past its grace stays while an older run is live, and goes when none is", async (t) => {
const f = await fixture(t);
const old = f.retiring(NOW - MIN);
const lost = f.retiring(NOW - MIN, { inList: false });
const record = f.seed({ issuedAt: NOW - HOUR, retiring: [clean(old), clean(lost)] });
f.state.keys = f.state.keys.filter((k) => k !== lost.key);
const me = processIdentity(process.pid);
const live = () => [{ phase: "running", supervisor: me }];
// This process is live (read from /proc); the seam dates its start before the rotation.
const retireGuard = { startedAt: () => NOW - 2 * HOUR };
// A lost key leaves the file even while held; the old key stays in the list and the file.
const o = f.opts({ runtimes: live, retireGuard });
await ensureReady(o);
assert.equal(o.stages[0], "locked");
assert.ok(f.state.keys.includes(old.key));
assert.deepEqual(f.read().retiring.map((r) => r.keyId), [old.keyId]);
// Held: the launch takes the fast path and leaves the old key.
const o2 = f.opts({ runtimes: live, retireGuard });
await ensureReady(o2);
assert.deepEqual(o2.stages, ["fast"]);
assert.ok(f.state.keys.includes(old.key));
// No runtimes source at all: held too, for every verb.
await retireKeys(f.opts());
await sweepKeys(f.opts());
assert.ok(f.state.keys.includes(old.key));
// A dead run from a failed launch does not hold it.
const o3 = f.opts({ runtimes: () => [deadRuntime(), { phase: "ended", supervisor: me }], retireGuard });
assert.equal((await ensureReady(o3)).action, "retired");
assert.equal(f.state.keys.includes(old.key), false);
assert.deepEqual(f.read().retiring, []);
assert.equal(f.read().key, record.key);
// Triage is not held: its retire drops a key past its grace with no runtimes source.
await issueKey({ configDir: f.configDir, system: f.system, now: () => NOW - 2 * DAY });
await rotateKey({ configDir: f.configDir, system: f.system, now: () => NOW - 2 * DAY });
const tOld = f.read("triage").retiring[0].key;
await retireKeys({ configDir: f.configDir, system: f.system, now: () => NOW });
assert.equal(f.state.keys.includes(tOld), false);
});
test("a due rotation waits while older runs hold MAX_RETIRING keys", async (t) => {
const f = await fixture(t);
const retiring = Array.from({ length: 16 }, () => clean(f.retiring(NOW - MIN)));
const record = f.seed({ issuedAt: NOW - 31 * DAY, retiring });
const me = processIdentity(process.pid);
const retireGuard = { startedAt: () => NOW - 32 * DAY };
const o = f.opts({ runtimes: () => [{ phase: "running", supervisor: me }], retireGuard });
const done = await ensureReady(o);
assert.equal(done.action, "held");
assert.deepEqual(o.stages, ["fast"]);
assert.equal(f.read().key, record.key);
// The terminal rotate still refuses, in worker words.
const e = await refused(rotateKey(f.opts({ runtimes: () => [{ phase: "running", supervisor: me }], retireGuard })), "worker-key-failed");
assert.match(e.message, /too many worker keys/);
});
test("keyStatus reads the worker file and its orphans", async (t) => {
const f = await fixture(t);
await ensureReady(f.opts());
const orphan = keyPrefix(f.configDir, "worker") + b64();
f.state.keys.push(orphan);
const s = await keyStatus({ configDir: f.configDir, system: f.system, purpose: "worker" });
assert.equal(s.proxy.state, "read");
assert.equal(s.proxy.holdsKey, true);
assert.deepEqual(s.proxy.orphanKeyIds, [id(orphan)]);
const triage = await keyStatus({ configDir: f.configDir, system: f.system });
assert.equal(triage.record, null);
assert.deepEqual(triage.proxy.orphanKeyIds, []);
// keyStatus returns the record itself (the Triage CLI prints only ids); scan the proxy half.
OUTPUTS.push(JSON.stringify(s.proxy), JSON.stringify(triage.proxy));
});
test("no output, error or receipt carries key material", () => {
assert.ok(SEEN.size > 10);
assert.ok(OUTPUTS.length > 10);
for (const text of OUTPUTS) assert.equal(leaks(text), false, "an output, error or receipt leaked a key");
});
// The Triage refusals keep their codes and words exactly as they were before purposes.
test("Triage refusals, receipts and fence keep their exact codes and words", async (t) => {
const f = await fixture(t);
const triage = (extra = {}) => ({ configDir: f.configDir, system: f.system, now: () => f.now, ...extra });
const exact = async (promise, code, message) => {
const e = await refused(promise, code);
if (message instanceof RegExp) assert.match(e.message, message); else assert.equal(e.message, message);
};
await exact(ensureKey({ configDir: f.configDir, system: { dataRoot: f.root } }), "triage-key-no-proxy", "the instance has no model proxy set up (proxy.json)");
await exact(rotateKey(triage()), "triage-key-not-issued", "no Triage key is issued; use mosaic triage key issue");
await issueKey(triage());
await exact(issueKey(triage()), "triage-key-issued", "a Triage key is already issued; use mosaic triage key rotate");
fs.chmodSync(f.triage.file, 0o644);
await exact(ensureKey(triage()), "triage-key-unreadable", /^the Triage key store is unreadable: /);
fs.chmodSync(f.triage.file, 0o600);
const sentinel = f.others.sentinel;
f.state.keys = f.state.keys.filter((k) => k !== sentinel);
await exact(ensureKey(triage()), "triage-key-proxy-open", "the proxy's key list has no sentinel for this instance, so it may be another proxy");
f.state.keys.push(sentinel);
f.state.onGet = (n) => { if (n % 2 === 0) f.state.keys.push(`unrelated-${n}`); };
await exact(rotateKey(triage()), "triage-key-raced", "the proxy's key list changed while this verb ran; run it again");
f.state.onGet = null;
f.state.ignorePut = true;
await exact(rotateKey(triage()), "triage-key-unconfirmed", "the proxy did not confirm the change; the instance proxy fence stays set, see mosaic proxy fence status");
f.state.ignorePut = false;
assert.match(fs.readFileSync(join(f.root, "proxy", "mosaic-keys.uncertain"), "utf8"), /"triage\.rotate"/);
fs.rmSync(join(f.root, "proxy", "mosaic-keys.uncertain"));
fs.chmodSync(f.receipts, 0o644);
await exact(rotateKey(triage()), "triage-key-receipts-unsafe", /; nothing was changed$/);
fs.chmodSync(f.receipts, 0o600);
await exact(rotateKey(triage({ hook: (s) => { if (s === "rename") throw new Error("stop"); } })), "triage-key-unwritten",
`the proxy list changed but ${f.triage.file} was not written (the Triage key store is unreadable: the key file cannot be written); fix the file's directory, then run mosaic triage key rotate again, which clears what this run left`);
await exact(rotateKey(triage({ hook: (s) => { if (s === "sync") throw new Error("stop"); } })), "triage-key-unsynced",
`${f.triage.file} was written but its directory could not be synced, so it may not survive a power loss; run mosaic triage key status`);
for (let i = 0; i < 20 && f.read("triage").retiring.length < 16; i++) await rotateKey(triage());
await exact(rotateKey(triage()), "triage-key-failed", "too many keys are retiring; run mosaic triage key retire once their grace has passed");
const ops = fs.readFileSync(f.receipts, "utf8").trim().split("\n").map((l) => JSON.parse(l).op);
assert.ok(ops.length > 3 && ops.every((op) => /^triage\.key\.(issue|rotate)$/.test(op)), ops.join(","));
});
+8 -5
View File
@@ -66,7 +66,7 @@ export const USAGE = `usage:
${PLUGINS_USAGE}
${CREDENTIALS_USAGE}
mosaic proxy fetch | status [--json] | fence status [--json] | fence clear --proxy-restarted | fence clear --restart [--business ID] | init [--generate-key] [--port N] [--business ID] [--rotate-grace-minutes N] [--first-output-seconds N]
mosaic proxy unit render|install|start|stop|restart|status [--json] | init-lock status|clear
mosaic proxy unit render|install|start|stop|restart|status [--json] | init-lock status|clear | ride --instance NAME | ride --off
${HOSTS_USAGE}
${GRANTS_USAGE}
${ORG_INSTANCES_USAGE}
@@ -677,15 +677,18 @@ async function bus(rest, io, deps) {
const priorBoot = readBootStatus(org.root);
try { recordBootOk(org.root, { business: businessId, file }); } catch { /* status is advisory */ }
const binding = readNotifyConfig(org.root, businessId);
const host = await startHost({ boot, business: businessId, notifier: binding ? { binding } : null, refuseStale: true, options: { devLocalOwner: Boolean(instance), pm: parsed.flags.has("--pm"), noLaunch: !launches } }).catch((e) => {
// A dev fixture serves only the organizations that use it; their launches carry no provider credential.
const orgDoc = instance ? JSON.parse(readFileSync(file, "utf8")) : undefined;
const launchEnv = instance ? fixtureLaunchEnv(orgDoc, io.env) : io.env;
// #1873 W2: the harness watch runs in the host with the env sessions launch from, unless the
// host launches nothing (--no-launch). deps.harnessWatch replaces its options in tests.
const harnessWatch = launches ? (deps.harnessWatch === undefined ? {} : deps.harnessWatch) : null;
const host = await startHost({ boot, business: businessId, notifier: binding ? { binding } : null, refuseStale: true, options: { devLocalOwner: Boolean(instance), pm: parsed.flags.has("--pm"), noLaunch: !launches }, harnessWatch: harnessWatch ? { env: launchEnv, ...harnessWatch } : null }).catch((e) => {
// org-defscan: the broker refused a definition that holds a credential it holds; the load was not good after all.
if (e.code === "definition-holds-credential") try { recordBootRefused(org.root, { business: businessId, file, reason: e.message, prior: priorBoot }); } catch { /* the refusal still throws */ }
throw e;
});
try {
// A dev fixture serves only the organizations that use it; their launches carry no provider credential.
const orgDoc = instance ? JSON.parse(readFileSync(file, "utf8")) : undefined;
const launchEnv = instance ? fixtureLaunchEnv(orgDoc, io.env) : io.env;
// U4a (#1873): agents get the task tools when the bus serves task verbs for the organization: a
// Vikunja tracker in the boot config, or the built-in store, which an organization with no
// tracker gets (the bus runtime makes the same choice, runtime.mjs trackerOf).
+17 -3
View File
@@ -17,6 +17,7 @@ import { systemConfigProblem } from "../../organization/src/system-config-proble
import { businessFilePath } from "../../organization/src/organization-config.mjs";
import { busOrganization } from "../../bus/src/organization-context.mjs";
import { BusError } from "../../bus/src/broker.mjs";
import { RIDE_LINES } from "../../model-proxy/src/settings.mjs";
import { CliError } from "./errors.mjs";
import { accessContextFor } from "./access.mjs";
@@ -57,6 +58,9 @@ function instanceProblem(problem, label) {
m = new RegExp(`^instance-config-mismatch: --instance (${INSTANCE}) names `).exec(body);
if (m) return `instance-config-mismatch: --instance ${m[1]} names its own config, but MOSAIC_CONFIG names another`;
if (body.startsWith("instance-config-mismatch: ")) return "instance-config-mismatch: --instance names its own config, but MOSAIC_CONFIG names another";
// proxy-ride refusals carry their fixed owner line from settings.mjs, never the child's text.
m = /^(proxy-ride-(?:invalid|owner-missing|conflict)): /.exec(body);
if (m) return `${m[1]}: ${RIDE_LINES[m[1]]}`;
if (body.startsWith("instance-name-invalid: ")) return `instance-name-invalid: an instance name must match /^${INSTANCE}$/`;
if (body.startsWith("configuration not found: ")) {
m = new RegExp(`\\(run (scripts/bootstrap\\.sh(?: --instance ${INSTANCE})?) to create it\\)$`).exec(body);
@@ -65,7 +69,14 @@ function instanceProblem(problem, label) {
return null;
}
export function loadSystem({ env = process.env, instance = null } = {}) {
// proxy-ride first slice (Jason 2026-11-21): on an instance whose proxy-ride.json names another
// instance, validate prints the owner's proxy with an `owner` object. Only worker runs use it
// (`ride: true`, packages/processing-control/src/container-cli.mjs). Every other caller, the
// bus boot and the Console among them, gets the system config without that proxy, so a riding
// instance's buses run exactly as an instance with no proxy runs. That holds while the owner is
// valid: validate resolves the owner on every call, so while proxy-ride.json exists an owner
// problem refuses every verb on the rider, buses and the Console included.
export function loadSystem({ env = process.env, instance = null, ride = false } = {}) {
const proc = spawnSync(process.execPath, [join(REPO, "scripts", "mosaic-config.mjs"), "validate", ...(instance === null ? [] : ["--instance", instance])], {
encoding: "utf8",
maxBuffer: 1024 * 1024,
@@ -76,11 +87,14 @@ export function loadSystem({ env = process.env, instance = null } = {}) {
const known = instanceProblem(String(proc.stderr ?? "").trim(), label);
throw new CliError(known ? `system config problem (mosaic-config exit ${proc.status}): ${known}` : systemConfigProblem(proc, env, label), 3);
}
let system;
try {
return JSON.parse(proc.stdout);
system = JSON.parse(proc.stdout);
} catch {
throw new CliError("system config: mosaic-config printed something that isn't JSON", 3);
}
if (!ride && system?.proxy?.owner) delete system.proxy;
return system;
}
export function rolesDir(env = process.env) {
@@ -207,7 +221,7 @@ export function bootConfig({ system, businessId, orgRoot = system.dataRoot, env
const config = {
dataRoot: orgRoot,
...(orgRoot === system.dataRoot ? {} : { instanceRoot: system.dataRoot, org: { id: b.id }, instanceDataRoot: system.dataRoot, instanceConfigDir: configDir(env) }),
...(system.proxy ? { proxy: system.proxy } : {}),
...(system.proxy && !system.proxy.owner ? { proxy: system.proxy } : {}),
businesses: { [b.id]: busOrganization(b, resolved) },
organization: {dir:configDir(env),rolesDir:rolesDir(env),system:vars},
launches: [],
+225
View File
@@ -0,0 +1,225 @@
// #1873 harness watch, slice W2 (reports/harness-watch-design.md section 1, "How often" and "Where
// results are kept"): the bus host's timer over the W1 watch (packages/plugins/src/harness-watch.mjs).
// It runs in the host process with the env sessions launch from.
//
// at start one full pass
// every statMs (5 minutes) a stat pass: the stat key of every harness module the loader gives
// (its optional watchKey({ env }), else its install roots), and a full pass when
// the key is not the last full pass's, or when dayMs (a day) has gone by since it
// halted/invalid while processing is halted or invalid (processingStatus), no pass at all; the
// first tick after it clears runs a full pass
//
// A harness module may ask for a stat pass now (packages/plugins/src/watch-request.mjs; claude-code's
// preflight asks when it has no verdict for its binary): it runs at once, or right after the pass
// running, and runs a full pass only when the key changed, as a tick does.
//
// One pass at a time: a tick that comes while a pass runs does nothing. A pass that throws is
// logged, with no path in the line, and the timer goes on; its key is not kept, so the next tick
// runs a full pass again. stop() clears the timer, aborts the pass running (the watch aborts the
// inspect it is in, which stops what it started) and waits for it; nothing is written after.
//
// The host is the one writer of two files under <org dataRoot>/state/:
// harness-watch.json { version: 1, checkedAt, harnesses: [record], refused, gone }: the current
// record per harness, as the watch returns it but without surfaces (their
// store is slice W6). Written whole to a temp file in the same directory, then
// renamed over the last one, 0600.
// harness-watch.jsonl one line per transition the watch reports, appended, never rewritten.
// The watch redacts its records (no host path, no env value). Both files are checked again before
// they are written: a line that would hold the scratch directory, the home directory, the data
// root or an env value of six characters or more is not written, and the pass is logged as failed.
// The timer adds no model call and no network call: the stat pass reads file metadata only, and a
// full pass is the watch's (its modules run their binaries with no network where the host allows).
import { appendFileSync, chmodSync, closeSync, fsyncSync, mkdirSync, mkdtempSync, openSync, readFileSync, renameSync, rmSync, writeSync } from "node:fs";
import { homedir, tmpdir } from "node:os";
import { join } from "node:path";
import { loadInstalled } from "../../plugins/src/loader.mjs";
import { watchHarnesses, watchStateFile } from "../../plugins/src/harness-watch.mjs";
import { statKey } from "../../plugins/src/stat-key.mjs";
import { onWatchRequest } from "../../plugins/src/watch-request.mjs";
import { processingStatus } from "../../processing-control/src/state.mjs";
export const STAT_MS = 5 * 60 * 1000;
export const DAY_MS = 24 * 60 * 60 * 1000;
export const WATCH_VERSION = 1;
// The bus's plugins verb reads this file (readWatchView in the same module as watchStateFile).
export const watchFile = watchStateFile;
export const watchLog = (dataRoot) => join(dataRoot, "state", "harness-watch.jsonl");
const VALUE_MIN = 6;
const realClock = Object.freeze({
now: () => Date.now(),
setInterval: (fn, ms) => setInterval(fn, ms),
clearInterval: (t) => clearInterval(t),
});
// The stat key of every harness the loader gives: per module, [name, its key]. A module's key is
// its watchKey({ env }) when it has one, else the stat key of its installRoots({ env }), else null.
// A module that comes or goes changes the key too. The key holds paths; it is compared in memory
// only.
export async function harnessKeys({ dataRoot, env }) {
const loaded = await loadInstalled("harness", { dataRoot });
const out = [];
for (const { name, module } of loaded.modules) {
let key = null;
try {
if (typeof module.watchKey === "function") key = module.watchKey({ env });
else if (typeof module.installRoots === "function") key = statKey(module.installRoots({ env }));
} catch (e) {
key = ["error", typeof e?.code === "string" ? e.code : "threw"];
}
out.push([name, typeof key === "string" || Array.isArray(key) ? key : null]);
}
out.push(["refused", loaded.refused.map((r) => r?.name ?? null)]);
return JSON.stringify(out);
}
// Starts the timer and its first pass. dataRoot is the organization's: the state files go under it.
// pluginRoot (W3) is where the harness plugins load from: the instance root when there is one, as
// the launcher loads them (launcher.mjs, system.instanceRoot ?? dataRoot); it defaults to dataRoot. Returns { stop, settled }: stop() ends it (a promise), and
// settled() is the pass running, or a resolved promise, for tests and for stop.
export function startHarnessWatch({
dataRoot,
pluginRoot = dataRoot,
env = process.env,
log = () => {},
clock = realClock,
processing = () => processingStatus(dataRoot),
watch = watchHarnesses,
keys = () => harnessKeys({ dataRoot: pluginRoot, env }),
statMs = STAT_MS,
dayMs = DAY_MS,
timeoutMs = undefined,
scratchRoot = () => tmpdir(),
requests = onWatchRequest,
} = {}) {
if (typeof dataRoot !== "string" || !dataRoot) throw new TypeError("startHarnessWatch needs the organization's dataRoot");
if (typeof pluginRoot !== "string" || !pluginRoot) throw new TypeError("startHarnessWatch needs a plugin root");
const control = new AbortController();
let stopped = false;
let running = null;
let held = false;
let again = false;
let needFull = true;
let lastKey = null;
let lastFull = null;
let last = readRecords(dataRoot);
async function pass() {
let status;
try { status = processing(); } catch { status = { invalid: true }; }
if (status?.halted || status?.invalid) {
if (!held) log("harness watch: processing is halted or invalid, so passes wait for it");
held = true;
return;
}
if (held) { held = false; needFull = true; }
const key = await keys();
if (stopped) return;
const now = clock.now();
if (!needFull && key === lastKey && lastFull !== null && now - lastFull < dayMs) return;
const scratch = mkdtempSync(join(scratchRoot(), "mosaic-watch-"));
try {
const result = await watch({ last, env, scratch, dataRoot: pluginRoot, now: () => new Date(clock.now()), signal: control.signal, ...(timeoutMs ? { timeoutMs } : {}) });
if (stopped || result?.stopped) return;
store(dataRoot, result, { env, scratch, pluginRoot, at: new Date(now).toISOString() });
last = result.records;
lastKey = key;
lastFull = now;
needFull = false;
} finally {
rmSync(scratch, { recursive: true, force: true });
}
}
function tick() {
if (stopped || running) return;
running = pass()
.catch((e) => { if (!stopped) log(`harness watch: a pass failed (${failure(e)}); the timer goes on`); })
.finally(() => {
running = null;
if (again && !stopped) { again = false; tick(); }
});
}
// A request runs a tick now, or once right after the pass running.
function request() {
if (stopped) return;
if (running) again = true;
else tick();
}
const timer = clock.setInterval(tick, statMs);
timer?.unref?.();
const unlisten = typeof requests === "function" ? requests(request) : null;
tick();
return Object.freeze({
settled: () => running ?? Promise.resolve(),
async stop() {
if (!stopped) {
stopped = true;
clock.clearInterval(timer);
if (typeof unlisten === "function") unlisten();
control.abort();
}
await (running ?? Promise.resolve());
},
});
}
// What failed, without the error's text, which can name a host path.
const failure = (e) => (typeof e?.code === "string" && /^[A-Za-z0-9_-]{1,40}$/.test(e.code) ? e.code : e?.name === "LeakError" ? "a state line held a host path or an env value" : "an error");
// The records of the last harness-watch.json, or none when it is missing or not this shape.
function readRecords(dataRoot) {
try {
const doc = JSON.parse(readFileSync(watchFile(dataRoot), "utf8"));
return doc?.version === WATCH_VERSION && Array.isArray(doc.harnesses) ? doc.harnesses.filter((r) => r && typeof r === "object" && typeof r.harness === "string") : [];
} catch {
return [];
}
}
function store(dataRoot, result, { env, scratch, pluginRoot = dataRoot, at }) {
const records = (Array.isArray(result?.records) ? result.records : []).map(({ surfaces, ...rest }) => rest);
const doc = { version: WATCH_VERSION, checkedAt: at, harnesses: records, refused: Array.isArray(result?.refused) ? result.refused : [], gone: Array.isArray(result?.gone) ? result.gone : [] };
const transitions = Array.isArray(result?.transitions) ? result.transitions : [];
const leaks = leakChecker({ env, scratch, dataRoot, pluginRoot });
if (leaks(doc) || transitions.some(leaks)) throw Object.assign(new Error("leak"), { name: "LeakError" });
const text = `${JSON.stringify(doc, null, 2)}\n`;
const lines = transitions.map((t) => `${JSON.stringify(t)}\n`).join("");
const dir = join(dataRoot, "state");
mkdirSync(dir, { recursive: true, mode: 0o700 });
const file = watchFile(dataRoot);
const tmp = `${file}.${process.pid}.${Date.now()}.tmp`;
try {
const fd = openSync(tmp, "wx", 0o600);
try { writeSync(fd, text); fsyncSync(fd); } finally { closeSync(fd); }
renameSync(tmp, file);
} catch (e) {
rmSync(tmp, { force: true });
throw e;
}
chmodSync(file, 0o600);
if (lines) appendFileSync(watchLog(dataRoot), lines, { mode: 0o600 });
}
// Whether a value holds a path the host knows (the scratch directory, the home directory, the data
// root) or an env value of VALUE_MIN characters or more, the host's or the launch env's, in any of
// its strings. Names the watch sets itself are not text and are skipped: a harness or module name
// (an env value may name the harness), a lookup step, a state, a fit, a via, a time or a digest.
const NOT_TEXT = new Set(["harness", "gone", "name", "kept", "source", "code", "state", "fit", "via", "at", "checkedAt", "changedAt", "digest", "lastDigest"]);
export function leakChecker({ env, scratch, dataRoot, pluginRoot = null }) {
const needles = new Set();
const add = (v) => { if (typeof v === "string" && v.length >= VALUE_MIN) needles.add(v); };
for (const p of [scratch, dataRoot, pluginRoot, homedir()]) add(p);
for (const source of [env ?? {}, process.env]) for (const v of Object.values(source)) add(v);
const holds = (text) => { for (const n of needles) if (text.includes(n)) return true; return false; };
const walk = (v, key) => {
if (typeof v === "string") return !NOT_TEXT.has(key) && holds(v);
if (Array.isArray(v)) return v.some((x) => walk(x, key));
if (v && typeof v === "object") return Object.entries(v).some(([k, x]) => walk(x, k));
return false;
};
return (value) => walk(value, null);
}
+12 -1
View File
@@ -22,6 +22,7 @@ import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { CliError } from "./errors.mjs";
import { cmdlineOf, startTimeOf } from "../../seat/src/proc.mjs";
import { startHarnessWatch } from "./harness-watch-host.mjs";
const BROKER = fileURLToPath(new URL("../../bus/src/process.mjs", import.meta.url));
const NOTIFIER = fileURLToPath(new URL("./notifier-process.mjs", import.meta.url));
@@ -175,7 +176,10 @@ export function watchChildren(children, onDeath) {
// {binding, base?, pollMs?}; base and pollMs exist for the tests, and the
// command line never sets them, nor requestTimeoutMs. Resolves once both
// children are up.
export async function startHost({ boot, business, notifier = null, options = null, refuseStale = false, bootTimeoutMs = BOOT_TIMEOUT_MS, requestTimeoutMs = REQUEST_TIMEOUT_MS, log = (l) => process.stderr.write(`mosaic-bus: ${l}\n`) }) {
// harnessWatch (#1873 W2): { env, ...options } starts the harness watch's timer in this process
// once the host is up (harness-watch-host.mjs), with env the environment sessions launch from; the
// host's close stops it first. null (the default) starts none.
export async function startHost({ boot, business, notifier = null, options = null, refuseStale = false, bootTimeoutMs = BOOT_TIMEOUT_MS, requestTimeoutMs = REQUEST_TIMEOUT_MS, log = (l) => process.stderr.write(`mosaic-bus: ${l}\n`), harnessWatch = null }) {
const dataRoot = boot.dataRoot;
// A host started before host.lock existed holds only host.json.
const prior = readHostState(dataRoot);
@@ -372,6 +376,13 @@ export async function startHost({ boot, business, notifier = null, options = nul
log(`${name} exited (${signal ?? code}); stopping the host`);
close(1);
});
// #1873 W2: the harness watch's timer, stopped before anything else when the host closes.
if (harnessWatch) {
// W3: plugins load from the instance root when there is one, as the launcher loads them; the
// state files stay under the organization's dataRoot.
const watch = startHarnessWatch({ log, pluginRoot: boot.instanceRoot ?? dataRoot, ...harnessWatch, dataRoot });
hooks.unshift(() => watch.stop());
}
// #1873 CB3: the copies the broker's boot proved live, names only; anything malformed counts as none.
const copyBound = {};
+80 -11
View File
@@ -215,12 +215,19 @@ const relayed = (failed) => failed && (typeof failed.message === "string" ? { ..
// and names the module rather than escaping the host. What it threw is named by its system code
// or its class, never its message, which the host didn't write (#1873).
function call(name, what, fn) {
try {
return fn();
} catch (e) {
const failed = (e) => {
const thrown = quietMessage(e, "a host file") ?? `threw ${/^[A-Za-z][A-Za-z0-9]{0,39}$/.test(String(e?.name)) ? e.name : "a value"} without a system code`;
throw new Refusal("harness-failed", `harness ${name} ${what}: ${thrown}`);
};
let out;
try {
out = fn();
} catch (e) {
failed(e);
}
// #1873 pi-unpin re-check 7: a module may answer with a promise (pi's preflight runs its fit
// check as child processes). The caller awaits it; a rejection is refused as a throw is.
return typeof out?.then === "function" ? Promise.resolve(out).then(undefined, failed) : out;
}
// host: startHost()'s handle. tracker: true when the broker has task verbs for
@@ -302,6 +309,11 @@ export function createLauncher({
let ticking = false;
const stopping = new Set();
const refusedThreads = new Map();
// #1873 chat-fail F2: the refusal and message each thread session's plain line was asked for.
const noticedThreads = new Map();
// #1873 F2: the last refusal seen per thread start, and the last failed post of its line.
const refusalSeen = new Map();
const noticeFailed = new Map();
const startedFor = new Map();
// Row 271 (264d): each clear's stop time (by op), and each clear refusal logged once (by op and code).
const clearStops = new Map();
@@ -449,14 +461,11 @@ export function createLauncher({
for (const m of [primary, fallback].filter(Boolean)) {
if (!on.includes(m.harness)) throw new Refusal("harness-off", `harness ${m.harness} isn't switched on in plugins.harness`);
}
// One harness through a fresh loader, so a refusal never leaves an earlier load's module in use.
const harnessOf = async (m) => {
const loader = makeLoader({ dataRoot:system.instanceRoot ?? dataRoot, log: (entry) => record({ ...entry, instance }) });
const { refused } = await loader.load({ harness: [m.harness] });
const { refused, module } = await freshHarness(m, (entry) => record({ ...entry, instance }));
// #1873: a loader message can quote the module's path or its load error; only a
// missing-export one, which names the export, is relayed. Any other refusal names its code.
if (refused.length) throw new Refusal("harness-refused", `harness ${m.harness}: ${refused.map((r) => r.code === "missing-export" ? `${r.code}: ${relay(r.message)}` : String(r.code)).join("; ")}`);
const module = loader.modules("harness")[0];
const levels = call(m.harness, "levels", () => module.levels());
if (!levels.includes(m.thinking)) throw new Refusal("thinking-unsupported", `harness ${m.harness} takes thinking ${levels.join(", ")}, not ${m.thinking}`);
return module;
@@ -551,7 +560,7 @@ export function createLauncher({
const pickRoute = async (proxyOf) => {
let sessionProxy = await proxyOf(primary);
// #1873: a preflight's message is the module's own text, relayed with host paths replaced.
const failed = relayed(call(primary.harness, "preflight", () => harness.preflight({ env: sessionProxy ? sessionEnv(env, "preflight", sessionProxy) : env, proxy: sessionProxy })));
const failed = relayed(await call(primary.harness, "preflight", () => harness.preflight({ env: sessionProxy ? sessionEnv(env, "preflight", sessionProxy) : env, proxy: sessionProxy })));
if (failed) {
if (!fallback) throw new Refusal(failed.code, failed.message);
// The fallback only on a failed primary preflight, and only at launch (row 65). Any refusal of
@@ -561,7 +570,7 @@ export function createLauncher({
throw e;
});
sessionProxy = await proxyOf(fallback);
const also = relayed(call(fallback.harness, "preflight", () => second.preflight({ env: sessionProxy ? sessionEnv(env, "preflight", sessionProxy) : env, proxy: sessionProxy })));
const also = relayed(await call(fallback.harness, "preflight", () => second.preflight({ env: sessionProxy ? sessionEnv(env, "preflight", sessionProxy) : env, proxy: sessionProxy })));
if (also) throw new Refusal(also.code, `fallback ${fallback.harness}: ${also.message} (primary ${primary.harness}: ${failed.message})`);
[model, harness, route, routeReason] = [fallback, second, "fallback", failed.code];
}
@@ -735,7 +744,7 @@ export function createLauncher({
if (store) compactions(store, model.harness, harness, { instance, thread, session, run });
const toolSocket = join(runDir, "tools.sock");
const turnMarker = join(runDir, "turn.done");
const version = harnessVersion !== undefined ? harnessVersion : call(model.harness, "version", () => harness.version({ env }));
const version = harnessVersion !== undefined ? harnessVersion : await call(model.harness, "version", () => harness.version({ env }));
// Row 137 (#1631): a role and thread holding an approved, unspent merge decision get land_work.
// The broker checks the decision again on every land; this only decides what the bundle shows.
// Row 156 (#1651): a non-thread instance's land and landing come from one landHolders read, so a
@@ -896,7 +905,13 @@ export function createLauncher({
}
children.set(run, exited);
const outputWatch = admittedProxy ? watchModelOutput({ runDir, armAtFirstTurn: true, keyId: modelKeyId, seconds: system.proxy.firstOutputSeconds ?? 120,
stop: () => host.op({ op: 'modelStall', business: b.id, role: instance, run, keyId: modelKeyId }), receipt: record }) : null;
stop: async () => {
const out = await host.op({ op: 'modelStall', business: b.id, role: instance, run, keyId: modelKeyId });
// #1873 chat-fail F2: the message the stopped turn was on gets one plain line, as a failed turn's.
try { await host.op({ op: 'failureNotice', record: { business: b.id, run, code: 'proxy-stall' } }); }
catch (err) { log(`run ${run}: failure line refused: ${err.code ?? 'failed'}`); }
return out;
}, receipt: record }) : null;
exited.then(async ({ code, signal }) => {
outputWatch?.cancel();
let receipt;
@@ -1066,8 +1081,30 @@ export function createLauncher({
await start({ instance: t.instance, launchedBy: b.human, ...where, legacy: t.legacy, legacyThread: legacyThreads.has(t.thread), ...(await check(t.instance, where)) });
startedFor.set(key, signature);
refusedThreads.delete(key);
noticedThreads.delete(key);
refusalSeen.delete(key);
noticeFailed.delete(key);
} catch (e) {
const code = e instanceof Refusal ? e.code : "launch-failed";
// #1873 chat-fail F2: the person who wrote gets one plain line per message the refusal
// leaves unanswered. Asked once per code and message; the bus keeps it to one per message.
// launch-failed can be a lost race that the next try wins, so it gets a line only when
// the same message is refused with it twice in a row. A post that fails is tried again
// on the next tick, and its failure is logged once.
const noticed = `${code}|${signature}`;
const repeated = refusalSeen.get(key) === noticed;
refusalSeen.set(key, noticed);
if (noticedThreads.get(key) !== noticed && (code !== "launch-failed" || repeated)) {
try {
await host.op({ op: "failureNotice", record: { business: b.id, thread: t.thread, session: t.session, code } });
noticedThreads.set(key, noticed);
noticeFailed.delete(key);
} catch (err) {
const failed = `${noticed}|${err.code ?? "failed"}`;
if (noticeFailed.get(key) !== failed) log(`thread ${t.thread} session ${t.session}: failure line refused: ${err.code ?? "failed"}`);
noticeFailed.set(key, failed);
}
}
// Logged once per code, not once per tick.
if (refusedThreads.get(key) === code) return;
refusedThreads.set(key, code);
@@ -1637,6 +1674,38 @@ export function createLauncher({
clearTimeout(timer);
}
// One harness ({ harness: name }) through a fresh loader, so a refusal never leaves an earlier
// load's module in use: { refused, module }, module null when refused. A launch (harnessOf) and
// the start's fit warm-up below (#1873 re-check 8) both load through here.
async function freshHarness(m, log) {
const loader = makeLoader({ dataRoot:system.instanceRoot ?? dataRoot, log });
const { refused } = await loader.load({ harness: [m.harness] });
return { refused, module: refused.length ? null : loader.modules("harness")[0] };
}
// #1873 pi-unpin re-check 8: each switched-on harness module with ready() (pi's fit check) starts
// it here, in the background, so the first launch joins the check in flight instead of starting
// it. Nothing waits on it: a launch still awaits its own preflight, which shares this check or
// reads its pass. A warm-up that fails or throws is dropped; it caches nothing, and the launch's
// preflight runs the check again. Re-check 9: a harness plugin the warm-up can't load leaves one
// launch log entry, by its code only (a launch that loads it records the message).
function warmHarnesses() {
const on = Array.isArray(b.plugins?.harness) ? b.plugins.harness : [];
for (const name of on) {
(async () => {
let logged = false;
const refusal = (entry) => {
if (logged || entry?.event !== "plugin-refused") return;
logged = true;
record({ event: "plugin-refused", kind: entry.kind, name: entry.name, code: entry.code, during: "warm-up" });
};
const { module } = await freshHarness({ harness: name }, refusal);
if (typeof module?.ready === "function") await module.ready({ env });
})().catch(() => {});
}
}
if (launches) warmHarnesses();
host.beforeClose(close);
return { business: b, registry, listen, launchPm, fromSession, tick, close };
}
+41 -1
View File
@@ -1,12 +1,15 @@
import { lstatSync } from 'node:fs';
import { join } from 'node:path';
import { configDir } from '../../organization/src/organization-config.mjs';
import { createProxyLock } from '../../bus/src/proxy-lock.mjs';
import { loadManagementClient } from '../../model-proxy/src/client.mjs';
import { CliError } from './errors.mjs';
import { loadSystem } from './config.mjs';
import { orgSelector } from './org-selector.mjs';
import { refuseInsideAgent } from './transport.mjs';
import { fetchPinned } from '../../model-proxy/src/fetch.mjs';
import { validManagementKey, proxyTiming } from '../../model-proxy/src/common.mjs';
import { RIDE_LINES, RIDE_NAME, readRide, createRide, removeRide } from '../../model-proxy/src/settings.mjs';
import { createProxyAdmin } from '../../model-proxy/src/admin-control.mjs';
import { listOrgs } from '../../organization/src/org-registry.mjs';
import { busesStopped, instanceFromConfig } from './proxy-admin-host.mjs';
@@ -91,13 +94,50 @@ async function setupCommand(verb, rest, io, { system, deps }) {
throw failure('proxy-usage', 4);
}
// proxy-ride first slice (Jason 2026-11-21): `mosaic proxy ride --instance NAME` writes this
// instance's proxy-ride.json, then loads the system config exactly as a worker run loads it; any
// refusal removes the file again, so a file the loader refuses is never left behind. `--off`
// removes the file without loading the config, so it works when the owner is gone. Neither form
// changes the owner instance, its proxy or any key.
const RIDE_CODES = Object.keys(RIDE_LINES);
const rideFailure = (code) => Object.assign(new CliError(`${code}: ${RIDE_LINES[code]}`, 2), { code });
const rideCode = (error) => RIDE_CODES.find(code => error?.code === code || new RegExp(`(?:^|: )${code}: `).test(error?.message ?? ''));
async function rideCommand(rest, io, { validate }) {
const dir = configDir(io.env);
if (rest.length === 1 && rest[0] === '--off') {
let removed;
try { removed = removeRide(dir); } catch { throw rideFailure('proxy-ride-invalid'); }
return print(io, { riding: null, removed });
}
if (rest.length !== 2 || rest[0] !== '--instance') throw failure('proxy-usage', 4);
const name = rest[1];
if (!RIDE_NAME.test(name)) throw rideFailure('proxy-ride-invalid');
let current;
try { current = readRide(dir); } catch (error) { throw rideFailure(rideCode(error) ?? 'proxy-ride-invalid'); }
if (current && current.instance !== name) {
throw Object.assign(new CliError(`proxy-usage: this instance already rides instance ${current.instance}; run mosaic proxy ride --off first`, 4), { code: 'proxy-usage' });
}
let undo = () => {};
if (!current) {
try { lstatSync(join(dir, 'proxy.json')); throw rideFailure('proxy-ride-conflict'); }
catch (error) { if (error instanceof CliError) throw error; if (error.code !== 'ENOENT') throw rideFailure('proxy-ride-conflict'); }
try { undo = createRide(dir, name); } catch { throw rideFailure('proxy-ride-invalid'); }
}
let system;
try { system = validate(); }
catch (error) { undo(); const code = rideCode(error); if (code) throw rideFailure(code); throw error; }
if (system?.proxy?.owner?.instance !== name || !Number.isInteger(system.proxy.port)) { undo(); throw rideFailure('proxy-ride-owner-missing'); }
return print(io, { riding: name, port: system.proxy.port });
}
export async function proxyCommand(args, io, { system, callFor, prompt = promptProxyKey, fetch = fetchPinned, ...deps }) {
const [verb, ...rest] = args;
if (args.includes('--tester')) throw failure('human-required', 3);
// #1873: every proxy verb, read or write, runs only from a human shell.
if (['init', 'fetch', 'fence', 'status', 'unit', 'init-lock'].includes(verb)) {
if (['init', 'fetch', 'fence', 'status', 'unit', 'init-lock', 'ride'].includes(verb)) {
try { refuseInsideAgent(io.env); } catch { throw failure('human-required', 3); }
}
if (verb === 'ride') return rideCommand(rest, io, { validate: deps.rideValidate ?? (() => loadSystem({ env: io.env, ride: true })) });
if (['status', 'unit', 'init-lock'].includes(verb)) return setupCommand(verb, rest, io, { system, deps: { ...deps, fetch } });
if (verb === 'fence' && rest[0] === 'clear' && rest.includes('--restart')) {
// #1873: restart the proxy unit, then send the restart evidence to the bus, which re-proves it
-294
View File
@@ -1,294 +0,0 @@
// #1873 triage-key: Triage's own model proxy client key, issued from the owner's terminal
// (design report reports/triage-route-design.md slice 3, Jason's ruling 2026-11-18).
//
// <configDir>/secrets/triage/model (directory 0700, file 0600, the running user's)
// { version, key, keyId, issuedAt, rotateBy, retiring: [{ key, keyId, issuedAt, retireAt }] }
//
// The key is `mst-<instance tag>-triage-<43 base64url>`, where the tag is the first 12 hex of
// sha256(configDir), the tag the instance's own `msk-` sentinel carries. An org bus reconciles only
// its own `mso-<tag>-<org>-key-` names and a no-org bus only `msk-<tag>-`, so neither touches it.
//
// Every proxy write runs under the instance proxy lock (packages/bus/src/proxy-lock.mjs), the lock
// org buses take: read the list, check this instance's sentinel, fence, PUT, read back, confirm.
// The proxy changes first and the file second, as the bus does for an agent key. A crash between
// leaves a key in the Triage namespace that the file does not hold; the next issue, rotate or retire
// removes it, and status names it. A crash after the PUT and before the read-back leaves the lock's
// uncertain fence, which every proxy writer on the instance refuses until a human clears it
// (`mosaic proxy fence clear --proxy-restarted`).
//
// Nothing runs on a timer. rotate records when the old key's grace ends; the key stays in the proxy
// until a later retire, rotate or issue removes it, and status says when that is due. The same goes
// for rotateBy: status marks it due; only rotate replaces the key.
//
// Nothing here prints, logs or records a key. Receipts and output carry key ids only (the first 12
// hex of sha256(key), the id the bus uses). Messages name files and options, never values.
// The Console reads the key at each Triage spawn through triage-key-file.mjs, never this module.
import { closeSync, constants, fchmodSync, fsyncSync, lstatSync, openSync, readdirSync, renameSync, unlinkSync, writeSync } from "node:fs";
import { randomBytes } from "node:crypto";
import { join } from "node:path";
import { isDeepStrictEqual } from "node:util";
import { createProxyLock } from "../../bus/src/proxy-lock.mjs";
import { loadManagementClient } from "../../model-proxy/src/client.mjs";
// #1873 triage-wiring: the key file's reader lives in triage-key-file.mjs (node built-ins only), so
// the Console reads the key without this module's proxy client. Re-exported here unchanged.
import { KEY_VERSION, MAX_RETIRING, SECRET, TriageKeyError, checkDir, checkFile, instanceTag, keyIdOf, keyPaths, keyPrefix, readKey, unreadable, validRecord } from "./triage-key-file.mjs";
export { KEY_VERSION, TriageKeyError, instanceTag, keyIdOf, keyPaths, keyPrefix, readKey, validRecord };
// The org key lifetime and the rotate grace default. The bus holds both as literals
// (packages/bus/src/model-keys.mjs #issue: 30 days to rotateBy; packages/bus/src/runtime.mjs:
// `proxy.rotateGraceMinutes ?? 120`) and exports neither, and this lane may not change the bus.
// packages/cli/tests/triage-key.test.mjs drives the bus's own ModelKeys with the same clock and
// proves these give the same rotateBy and retireAt, so a change on either side goes red there.
// The grace itself is read from proxy.json first, where the bus reads it.
export const KEY_LIFETIME_DAYS = 30;
export const DEFAULT_ROTATE_GRACE_MINUTES = 120;
const sentinelPrefix = (configDir) => `msk-${instanceTag(configDir)}-sentinel-`;
// Any name under the prefix is Triage's to remove, well formed or not, as the bus treats its own
// `msk-<tag>-` names. Only a well-formed key may be stored in the file.
const inNamespace = (configDir) => (k) => typeof k === "string" && k.startsWith(keyPrefix(configDir));
// Both directories, checked (and the triage one made) before any proxy write, so a store that cannot
// be written refuses while the proxy is still untouched.
function prepareDirs(configDir, uid) {
const { secrets, dir } = keyPaths(configDir);
if (!checkDir(secrets, uid, "the secrets directory", { create: false })) throw unreadable("the secrets directory is missing");
checkDir(dir, uid, "the triage secrets directory", { create: true });
}
// A temp file writeKey left when it was killed before its rename. It may hold a key.
const TEMP = /^\.model\.\d+\.[0-9a-f]{12}\.tmp$/;
const tempsIn = (dir) => { try { return readdirSync(dir).filter((n) => TEMP.test(n)); } catch { return []; } };
// Under the lock no other Triage writer runs, so every temp file left in the directory is dead.
function removeTemps(configDir) {
const { dir } = keyPaths(configDir);
for (const name of tempsIn(dir)) {
try { if (!lstatSync(join(dir, name)).isDirectory()) unlinkSync(join(dir, name)); }
catch { throw unreadable("a leftover temp file in the triage secrets directory cannot be removed"); }
}
}
// A directory sync that failed after the rename: the file is in place but may not be durable.
class NotSynced extends Error {}
// Atomic: a 0600 temp file in the same directory, fsync, rename, fsync of the directory.
function writeKey(configDir, record, uid, hook = () => {}) {
const { dir, file } = keyPaths(configDir);
prepareDirs(configDir, uid);
const temp = join(dir, `.model.${process.pid}.${randomBytes(6).toString("hex")}.tmp`);
let fd;
try { fd = openSync(temp, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600); }
catch { throw unreadable("the key file cannot be written"); }
try {
try { fchmodSync(fd, 0o600); writeSync(fd, `${JSON.stringify(record, null, 2)}\n`); fsyncSync(fd); }
finally { closeSync(fd); }
hook("rename");
renameSync(temp, file);
} catch { try { unlinkSync(temp); } catch {} throw unreadable("the key file cannot be written"); }
try {
hook("sync");
const dfd = openSync(dir, constants.O_RDONLY | constants.O_DIRECTORY);
try { fsyncSync(dfd); } finally { closeSync(dfd); }
} catch { throw new NotSynced(); }
}
// One line per write in the proxy lock's receipts file, ids only. `receiptsReady` runs under the
// lock before any change, so an unsafe receipts file refuses while nothing has changed yet.
const RECEIPTS_FLAGS = constants.O_WRONLY | constants.O_APPEND | constants.O_CREAT | constants.O_NOFOLLOW | constants.O_NONBLOCK;
const receiptsUnsafe = (why) => new TriageKeyError("triage-key-receipts-unsafe", `${why}; nothing was changed`);
function receiptsReady(stateDir, uid) {
let fd;
try { fd = openSync(join(stateDir, "mosaic-keys.receipts.jsonl"), RECEIPTS_FLAGS, 0o600); }
catch { throw receiptsUnsafe("the proxy receipts file cannot be opened"); }
try { checkFile(fd, uid, "the proxy receipts file", receiptsUnsafe); } finally { closeSync(fd); }
}
function receipt(stateDir, uid, entry) {
const unrecorded = new TriageKeyError("triage-key-unrecorded", "the change was made but its receipt line was not written; run mosaic triage key status");
let fd;
try { fd = openSync(join(stateDir, "mosaic-keys.receipts.jsonl"), RECEIPTS_FLAGS, 0o600); } catch { throw unrecorded; }
try {
checkFile(fd, uid, "the proxy receipts file", () => unrecorded);
writeSync(fd, `${JSON.stringify(entry)}\n`);
fsyncSync(fd);
} catch { throw unrecorded; }
finally { closeSync(fd); }
}
// The grace the bus uses: proxy.json's rotateGraceMinutes, or the bus default.
export const graceMinutes = (proxy) => proxy?.rotateGraceMinutes ?? DEFAULT_ROTATE_GRACE_MINUTES;
export const rotateByFor = (nowMs) => new Date(nowMs + KEY_LIFETIME_DAYS * 86400_000).toISOString().slice(0, 10);
export const retireAtFor = (nowMs, proxy) => new Date(nowMs + graceMinutes(proxy) * 60_000).toISOString();
const proxyFailure = (code) => {
const words = {
"proxy-lock-busy": "another proxy writer holds the instance proxy lock; try again when it ends, and if no writer is running, see mosaic proxy fence status",
"proxy-lock-uncertain": "the instance proxy lock is in a state this verb will not guess about; check <dataRoot>/proxy by hand",
"proxy-lock-invalid": "the instance proxy lock directory is missing or unsafe, or names another proxy",
"proxy-uncertain": "an earlier proxy write has an uncertain outcome; see mosaic proxy fence status",
"proxy-management-key-invalid": "the proxy management key file is missing or unsafe",
"proxy-client-invalid": "the proxy settings are not usable",
"proxy-client-down": "the proxy did not answer",
"proxy-client-refused": "the proxy refused the management request",
"proxy-client-redirect": "the proxy answered with a redirect",
"proxy-client-response": "the proxy gave an answer this verb cannot use",
};
return new TriageKeyError(code, words[code] ?? "the proxy write failed");
};
// opts.uid, opts.now and opts.hook(stage) are test seams; a stage hook may throw or kill the process.
function context({ configDir, system, uid, fileUid, now, hook }) {
if (!system?.proxy || !Number.isInteger(system.proxy.port)) throw new TriageKeyError("triage-key-no-proxy", "the instance has no model proxy set up (proxy.json)");
return {
configDir, dataRoot: system.dataRoot, proxy: system.proxy, stateDir: join(system.dataRoot, "proxy"),
uid: uid ?? process.getuid(), fileUid, now: now ?? Date.now, hook: hook ?? (() => {}),
};
}
function lockFor(c) {
try { return createProxyLock({ stateDir: c.stateDir, identity: `${c.configDir}:${c.proxy.port}` }); }
catch (e) { throw proxyFailure(e.code ?? "proxy-lock-invalid"); }
}
async function clientFor(c) {
try { return await loadManagementClient({ configDir: c.configDir, proxy: c.proxy }); }
catch (e) { throw proxyFailure(e.code ?? "proxy-management-key-invalid"); }
}
function checkSentinel(c, keys) {
const prefix = sentinelPrefix(c.configDir);
if (!keys.some((k) => k.startsWith(prefix) && SECRET.test(k.slice(prefix.length)))) {
throw new TriageKeyError("triage-key-proxy-open", "the proxy's key list has no sentinel for this instance, so it may be another proxy");
}
}
// Under the instance proxy lock: read the list, check the sentinel, run `plan`, write the result.
// `plan(keys, record)` returns { after, record, removed, added, done }; `record` null leaves the
// file alone. The proxy changes first, then the file, then one receipt line.
async function change(c, op, plan) {
const lock = lockFor(c);
const client = await clientFor(c);
try {
return await lock.run(async () => {
const before = await client.getKeys();
checkSentinel(c, before);
const current = readKey(c.configDir, { uid: c.uid, fileUid: c.fileUid });
const result = plan(before, current);
if (result.record) prepareDirs(c.configDir, c.uid);
removeTemps(c.configDir);
receiptsReady(c.stateDir, c.uid);
const after = result.after;
const removed = before.filter((k) => !after.includes(k));
const added = after.filter((k) => !before.includes(k));
// A key in another namespace never leaves the list.
if (removed.some((k) => !inNamespace(c.configDir)(k))) throw new TriageKeyError("triage-key-failed", "the change would remove a key outside the Triage namespace");
if (added.length || removed.length) {
// As the bus does: a list that moved since the first read is not overwritten.
if (!isDeepStrictEqual(await client.getKeys(), before)) throw new TriageKeyError("triage-key-raced", "the proxy's key list changed while this verb ran; run it again");
lock.fence({ org: null, op: `triage.${op}` });
await client.putKeys(after);
c.hook("put");
const confirmed = await client.getKeys();
if (!confirmed.some((k) => k.startsWith(sentinelPrefix(c.configDir))) || added.some((k) => !confirmed.includes(k)) || removed.some((k) => confirmed.includes(k))) {
throw new TriageKeyError("triage-key-unconfirmed", "the proxy did not confirm the change; the instance proxy fence stays set, see mosaic proxy fence status");
}
lock.confirm();
}
c.hook("write");
let unsynced = false;
if (result.record) {
try { writeKey(c.configDir, result.record, c.uid, c.hook); }
catch (e) {
if (e instanceof NotSynced) unsynced = true;
else throw new TriageKeyError("triage-key-unwritten", `the proxy list changed but ${keyPaths(c.configDir).file} was not written (${e.message}); fix the file's directory, then run mosaic triage key ${op} again, which clears what this run left`);
}
}
c.hook("receipt");
// A run that changed neither the proxy nor the file leaves no receipt.
if (added.length || removed.length || result.record) receipt(c.stateDir, c.uid, { op: `triage.key.${op}`, time: new Date(c.now()).toISOString(), addedKeyIds: added.map(keyIdOf), removedKeyIds: removed.map(keyIdOf) });
// The file is in place and the receipt written; only its durability is in doubt.
if (unsynced) throw new TriageKeyError("triage-key-unsynced", `${keyPaths(c.configDir).file} was written but its directory could not be synced, so it may not survive a power loss; run mosaic triage key status`);
return { ...result.done, addedKeyIds: added.map(keyIdOf), removedKeyIds: removed.map(keyIdOf) };
});
} catch (e) {
if (e instanceof TriageKeyError) throw e;
throw proxyFailure(typeof e?.code === "string" && /^proxy-[a-z-]+$/.test(e.code) ? e.code : "triage-key-failed");
}
}
const newKey = (configDir) => keyPrefix(configDir) + randomBytes(32).toString("base64url");
// Keys in the Triage namespace that the file does not hold: left by a crash between the PUT and the
// file write, or by a file that was removed by hand.
const orphansOf = (configDir, keys, record) => {
const held = record ? [record.key, ...record.retiring.map((r) => r.key)] : [];
return keys.filter((k) => inNamespace(configDir)(k) && !held.includes(k));
};
export async function issueKey(opts) {
const c = context(opts);
return change(c, "issue", (before, current) => {
if (current) throw new TriageKeyError("triage-key-issued", "a Triage key is already issued; use mosaic triage key rotate");
const orphans = orphansOf(c.configDir, before, null);
const key = newKey(c.configDir), nowMs = c.now();
const record = { version: KEY_VERSION, key, keyId: keyIdOf(key), issuedAt: new Date(nowMs).toISOString(), rotateBy: rotateByFor(nowMs), retiring: [] };
return { after: [...before.filter((k) => !orphans.includes(k)), key], record, done: { keyId: record.keyId, rotateBy: record.rotateBy } };
});
}
// Retiring keys whose grace has passed, or that the proxy no longer holds.
const dueOf = (record, keys, nowMs) => record.retiring.filter((r) => Date.parse(r.retireAt) <= nowMs || !keys.includes(r.key));
export async function rotateKey(opts) {
const c = context(opts);
return change(c, "rotate", (before, current) => {
if (!current) throw new TriageKeyError("triage-key-not-issued", "no Triage key is issued; use mosaic triage key issue");
const nowMs = c.now();
const orphans = orphansOf(c.configDir, before, current);
const due = dueOf(current, before, nowMs);
const keep = before.includes(current.key);
const retiring = [...current.retiring.filter((r) => !due.includes(r)),
...(keep ? [{ key: current.key, keyId: current.keyId, issuedAt: current.issuedAt, retireAt: retireAtFor(nowMs, c.proxy) }] : [])];
if (retiring.length > MAX_RETIRING) throw new TriageKeyError("triage-key-failed", "too many keys are retiring; run mosaic triage key retire once their grace has passed");
const key = newKey(c.configDir);
const record = { version: KEY_VERSION, key, keyId: keyIdOf(key), issuedAt: new Date(nowMs).toISOString(), rotateBy: rotateByFor(nowMs), retiring };
const drop = [...orphans, ...due.map((r) => r.key)];
return {
after: [...before.filter((k) => !drop.includes(k)), key], record,
done: { keyId: record.keyId, rotateBy: record.rotateBy, old: keep ? { keyId: current.keyId, retireAt: retiring.at(-1).retireAt } : { keyId: current.keyId, lost: true } },
};
});
}
export async function retireKeys(opts) {
const c = context(opts);
return change(c, "retire", (before, current) => {
const nowMs = c.now();
const orphans = orphansOf(c.configDir, before, current);
const due = current ? dueOf(current, before, nowMs) : [];
const record = current && due.length ? { ...current, retiring: current.retiring.filter((r) => !due.includes(r)) } : null;
const drop = [...orphans, ...due.map((r) => r.key)];
return { after: before.filter((k) => !drop.includes(k)), record, done: { droppedKeyIds: due.map((r) => r.keyId) } };
});
}
// Read only: the file, and when the proxy is set up, what its list holds. No lock, no write.
// The fence is checked by name only, so status writes nothing, not even the lock's identity file.
export async function keyStatus(opts) {
const { configDir, system, uid, fileUid } = opts;
const record = readKey(configDir, { uid, fileUid });
const proxy = { state: "not-set-up", fence: false, leftoverTemps: tempsIn(keyPaths(configDir).dir).length };
if (system?.proxy && Number.isInteger(system.proxy.port)) {
try { lstatSync(join(system.dataRoot, "proxy", "mosaic-keys.uncertain")); proxy.fence = true; }
catch (e) { if (e.code !== "ENOENT") proxy.fence = true; }
try {
const client = await loadManagementClient({ configDir, proxy: system.proxy });
const keys = await client.getKeys();
const prefix = sentinelPrefix(configDir);
proxy.state = keys.some((k) => k.startsWith(prefix) && SECRET.test(k.slice(prefix.length))) ? "read" : "no-sentinel";
proxy.holdsKey = record ? keys.includes(record.key) : null;
proxy.retiringHeld = record ? record.retiring.map((r) => keys.includes(r.key)) : [];
proxy.orphanKeyIds = orphansOf(configDir, keys, record).map(keyIdOf);
} catch (e) { proxy.state = "unreadable"; proxy.code = typeof e?.code === "string" && /^proxy-[a-z-]+$/.test(e.code) ? e.code : "proxy-client-response"; }
}
return { record, proxy };
}
+2 -2
View File
@@ -16,7 +16,7 @@
//
// mosaic triage key issue | rotate | retire | status
//
// Triage's own model proxy client key (packages/cli/src/triage-key.mjs). Human verbs as well,
// Triage's own model proxy client key (packages/bus/src/triage-key.mjs, shared with the bus). Human verbs as well,
// refused in an agent run before anything is read. No output names a key, only its id.
import { lstatSync } from "node:fs";
import { join } from "node:path";
@@ -24,7 +24,7 @@ import { fileURLToPath } from "node:url";
import { readRoute, RouteStoreError, writeRoute } from "../../webui/src/triage-route-store.mjs";
import { loadSystem } from "./config.mjs";
import { configDir } from "../../organization/src/organization-config.mjs";
import { issueKey, keyPaths, keyStatus, retireKeys, rotateKey, TriageKeyError } from "./triage-key.mjs";
import { issueKey, keyPaths, keyStatus, retireKeys, rotateKey, TriageKeyError } from "../../bus/src/triage-key.mjs";
import { CliError } from "./errors.mjs";
import { refuseInsideAgent } from "./transport.mjs";
+1
View File
@@ -105,6 +105,7 @@ async function start(t, { f, adapter }, { businessId = "acme", orgRoot, copyTran
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
threadsState: (m) => runtime.threadsState(m.business),
threadIdle: (m) => runtime.threadIdle(m.record),
failureNotice: (m) => runtime.failureNotice(m.record),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
@@ -0,0 +1,83 @@
// #1873 harness watch, slice W3: `mosaic bus start ORG --internal-host` starts the harness watch in
// the host (through deps.harnessWatch, a stand-in watch), with the instance root as the root the
// harness plugins load from and the organization's root for its state files, as the launcher loads
// them; `--no-launch` starts no watch. The host runs in this process and stops on SIGTERM, as the
// unit's stop does. Nothing is launched and no live port is used.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { spawnSync } from 'node:child_process';
import { existsSync, mkdtempSync, readFileSync, rmSync } from 'node:fs';
import { join, resolve } from 'node:path';
import { tmpdir } from 'node:os';
import * as org from '../../organization/src/org-service.mjs';
import { main } from '../src/cli.mjs';
import { watchFile } from '../src/harness-watch-host.mjs';
import { humanEnv } from './helpers.mjs';
const repo = resolve(import.meta.dirname, '../../..');
async function instance(t) {
const home = mkdtempSync(join(tmpdir(), 'w3-'));
t.after(() => rmSync(home, { recursive: true, force: true }));
const env = humanEnv({ HOME: home, MOSAIC_CONFIG: join(home, '.config/mosaic-w3/config.json') });
const boot = spawnSync(join(repo, 'scripts/bootstrap.sh'), ['--instance', 'w3'], { env, encoding: 'utf8' });
assert.equal(boot.status, 0, boot.stderr);
const ctx = { configDir: join(home, '.config/mosaic-w3'), dataRoot: join(home, '.mosaic-w3'), instance: 'w3', env, principal: { provider: 'local-dev', iss: 'local-dev', sub: 'local-dev-owner' } };
await org.createOrg(ctx, 'alpha', await org.templateRender('alpha', { fixture: 'local' }));
return { ctx, env, orgRoot: join(ctx.dataRoot, 'org-runtime', 'alpha') };
}
// Runs the host in this process until `until` says so, then stops it as SIGTERM does.
async function hostRun(args, env, deps, until) {
let out = '', err = '';
const io = { env, stdin: null, stdout: { write: (s) => { out += s; return true; } }, stderr: { write: (s) => { err += s; return true; } } };
const listeners = process.listenerCount('SIGTERM');
const running = main(['bus', 'start', 'alpha', '--internal-host', ...args], io, deps);
let finished = false;
running.finally(() => { finished = true; });
const end = Date.now() + 30000;
// The host prints its line before bus start takes SIGTERM, so the stop waits for its listener too.
while (!(out.includes('bus host up') && process.listenerCount('SIGTERM') > listeners && until()) && !finished && Date.now() < end) await new Promise((r) => setTimeout(r, 20));
const up = out.includes('bus host up');
if (!finished) process.emit('SIGTERM');
const code = await running;
return { code, out, err, up };
}
test('W3 bus start starts the harness watch in the host, loading plugins from the instance root and writing state under the organization root', { timeout: 90000 }, async (t) => {
const { ctx, env, orgRoot } = await instance(t);
const calls = [];
const scratch = mkdtempSync(join(tmpdir(), 'w3-scratch-'));
t.after(() => rmSync(scratch, { recursive: true, force: true }));
const record = { harness: 'pi', state: 'fit', via: 'inspect', found: true, source: 'PATH', version: '1.6.0', digest: 'b'.repeat(64), lastDigest: 'b'.repeat(64), fit: 'pass', definite: true, failed: [], line: 'pi 1.6.0 passed the fit check', checkedAt: '2026-11-20T10:00:00.000Z', changedAt: '2026-11-20T10:00:00.000Z' };
const harnessWatch = {
watch: async (opts) => { calls.push(opts); return { records: [record], transitions: [], refused: [], gone: [] }; },
keys: async () => 'k',
processing: () => ({ halted: false }),
scratchRoot: () => scratch,
};
const run = await hostRun([], env, { harnessWatch }, () => calls.length > 0 && existsSync(watchFile(orgRoot)));
assert.equal(run.up, true, run.err);
assert.match(run.out, /bus host stopped \(0\)/, run.err);
assert.equal(calls.length, 1, 'the start pass ran in the host');
assert.notEqual(orgRoot, ctx.dataRoot);
assert.equal(calls[0].dataRoot, ctx.dataRoot, 'the watch loads harness plugins from the instance root, as the launcher does');
assert.equal(calls[0].signal.aborted, true, 'the stop aborted the watch');
assert.deepEqual(JSON.parse(readFileSync(watchFile(orgRoot), 'utf8')).harnesses.map((r) => r.harness), ['pi'], 'its state file is under the organization root');
assert.equal(existsSync(watchFile(ctx.dataRoot)), false, 'and not under the instance root');
});
test('W3 bus start --no-launch starts no harness watch', { timeout: 90000 }, async (t) => {
const { env, orgRoot } = await instance(t);
const calls = [];
const harnessWatch = { watch: async (opts) => { calls.push(opts); return { records: [], transitions: [], refused: [], gone: [] }; }, keys: async () => { calls.push('keys'); return 'k'; }, processing: () => ({ halted: false }), scratchRoot: () => tmpdir() };
// A watch starts with the host and runs its start pass at once, so the host stays up 500 ms after
// its line before the stop.
let upAt = null;
const run = await hostRun(['--no-launch'], env, { harnessWatch }, () => { upAt ??= Date.now(); return Date.now() - upAt >= 500; });
assert.equal(run.up, true, run.err);
assert.match(run.out, /launches disabled/);
assert.match(run.out, /bus host stopped \(0\)/, run.err);
assert.deepEqual(calls, [], 'no watch, no key read');
assert.equal(existsSync(watchFile(orgRoot)), false);
});
@@ -99,6 +99,7 @@ async function start(t, { f, adapter }) {
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
threadsState: (m) => runtime.threadsState(m.business),
threadIdle: (m) => runtime.threadIdle(m.record),
failureNotice: (m) => runtime.failureNotice(m.record),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
+1
View File
@@ -30,6 +30,7 @@ const ops = {
credentialStatus: m => runtime.credentialStatus(m.business, m.role),
threadsState: m => runtime.threadsState(m.business),
threadIdle: m => runtime.threadIdle(m.record),
failureNotice: m => runtime.failureNotice(m.record),
mayLaunch: m => runtime.mayLaunch(m),
landHeld: m => runtime.landHeld(m.record),
landHolders: m => runtime.landHolders(m.business),
+2 -1
View File
@@ -16,7 +16,7 @@ import { pushWorking, fetchReview } from '../../../harness/src/git-tools.mjs';
import { callTool } from '../../../harness/src/tools.mjs';
import { toolServer } from '../../../harness/src/runner.mjs';
import { buildBundle } from '../../../harness/src/bundle.mjs';
import { HARNESS } from '../../../harness/tests/helpers.mjs';
import { HARNESS, harnessReady } from '../../../harness/tests/helpers.mjs';
import { ensureCheckouts } from '../../../seat/src/checkout.mjs';
import { newRun, Registry, readSessions, writeSessionFile, workspaceDir } from '../../../seat/src/session.mjs';
import { startTimeOf } from '../../src/host.mjs';
@@ -127,6 +127,7 @@ async function runtime() {
const cap = bus.bindLaunch(launch_record).cap;
roles[role] = { cap, run: id, workspace, runDir };
call(role, 'role.claim');
await harnessReady(); // #1873 re-check 7: buildBundle reads a passed fit check
const bundle = buildBundle({ dir: join(runDir, 'bundle'), resolved, contract: readFileSync(resolved.contract, 'utf8'),
arbiter: role === 'cto', tracker: Boolean(resolved.tracker), credentials: [], skillsDir: join(dataRoot, 'skills-enabled'), harness: HARNESS.pi,
harnessVersion: '0.85.1', session: { run: id, workspace, checkouts, toolSocket: join(runDir, 'tools.sock'), turnMarker: join(runDir, 'turn.done') } });
+14 -12
View File
@@ -66,35 +66,37 @@ exit 0
const env = { PATH: `${bin}:${process.env.PATH}`, HOME: process.env.HOME, TMPDIR: tmp, LANG: 'C.UTF-8', FAKE_DOCKER_RECORD: record, ...extra };
const r = spawnSync('bash', [join(checkout, 'scripts', script)], { env, encoding: 'utf8', timeout: 600000 });
t.diagnostic(JSON.stringify({ script, extra: Object.keys(extra), wallMs: Math.round(performance.now() - begin), status: r.status }));
const runs = existsSync(record) ? readFileSync(record, 'utf8').trim().split('\n').filter(Boolean).map(l => { const [file, kind, adapter] = l.split('\t'); return { file, kind, adapter }; }) : [];
const runs = existsSync(record) ? readFileSync(record, 'utf8').split('\n').filter(Boolean).map(l => { const [file, kind, adapter] = l.split('\t'); return { file, kind, adapter }; }) : [];
return { out: r.stdout + r.stderr, runs, tmp };
};
}
const fixtureRuns = (runs, tmp) => runs.every(r => r.file !== HOST_AUTH && r.file.startsWith(tmp + '/') && r.kind === 'regular file 600');
// worker-cutover: the suites export no PI_AUTH_FILE and compose mounts no credential file, so no
// compose run carries one, and none ever names the host's.
const noAuthRuns = runs => runs.every(r => r.file === '' && r.kind === 'unset');
test('test-release.sh activates through the mock adapter with a 0600 fixture auth file', t => {
const { out, runs, tmp } = suiteLane(t)('test-release.sh');
test('test-release.sh activates through the mock adapter with no auth file', t => {
const { out, runs } = suiteLane(t)('test-release.sh');
assert.ok(runs.length >= 3, `fault, healthy and repeat activations reach Compose\n${out}`);
assert.ok(fixtureRuns(runs, tmp), JSON.stringify(runs));
assert.ok(noAuthRuns(runs), JSON.stringify(runs));
assert.deepEqual([...new Set(runs.map(r => r.adapter))], ['mock'], 'ruling A: no pi adapter in the gate');
assert.match(out, /OK +healthy activation succeeds/); assert.match(out, /OK +fault-injected activation refuses/);
});
test('test-task.sh mounts only its fixture, uses only the mock adapter and prints the live skip twice', t => {
const { out, runs, tmp } = suiteLane(t)('test-task.sh');
test('test-task.sh passes no auth file, uses only the mock adapter and prints the live skip twice', t => {
const { out, runs } = suiteLane(t)('test-task.sh');
assert.ok(runs.length >= 1, out);
assert.ok(fixtureRuns(runs, tmp), JSON.stringify(runs));
assert.ok(noAuthRuns(runs), JSON.stringify(runs));
assert.deepEqual([...new Set(runs.map(r => r.adapter))], ['mock']);
assert.equal(out.split('\n').filter(l => l === LIVE_SKIP).length, 2, out);
assert.match(out, /OK +hello task succeeds with exact marker/);
});
test('M-T: MOSAIC_LIVE_PROVIDER=1 runs the real-model cases, which fail under a fixture auth file', t => {
test('M-T: MOSAIC_LIVE_PROVIDER=1 runs the real-model cases, which fail before Compose with no model proxy', t => {
const lane = suiteLane(t), auth = join(scratch(t), 'operator-auth.json'); writeFileSync(auth, '{}'); chmodSync(auth, 0o600);
const { out, runs } = lane('test-task.sh', { MOSAIC_LIVE_PROVIDER: '1', PI_AUTH_FILE: auth });
assert.ok(!out.includes(LIVE_SKIP), out);
// The fake answers a pi run with nothing: the fork base misses REMEMBERED, recall misses the name.
// worker-cutover: an operator's PI_AUTH_FILE no longer runs a worker. The suite's pi config has no
// model proxy next to it, so each pi launch refuses before Compose: the fork base and the recall fail.
assert.match(out, /FAIL fork base: teach succeeds/); assert.match(out, /FAIL recalled user name/);
assert.ok(runs.some(r => r.adapter === 'pi'), JSON.stringify(runs));
assert.ok(runs.every(r => r.file === auth), 'live mode keeps the operator\'s PI_AUTH_FILE');
assert.ok(runs.length >= 1 && runs.every(r => r.adapter === 'mock'), JSON.stringify(runs));
});
@@ -30,9 +30,10 @@ function fixture(t) {
const actual = { Id: 'sha256:' + 'a'.repeat(64), Os: 'linux', Architecture: 'amd64' };
function base(root) { return /^FROM (\S+)$/m.exec(readFileSync(join(root, 'Containerfile'), 'utf8'))[1]; }
function receipt(root) {
const pkg = JSON.parse(readFileSync(join(root, 'package.json'), 'utf8'));
// #1873 S5: the tag carries the pi version the lock resolves, never the package.json range.
const locked = JSON.parse(readFileSync(join(root, 'package-lock.json'), 'utf8')).packages['node_modules/@earendil-works/pi-coding-agent'].version;
const release = readFileSync(join(root, 'RELEASE'), 'utf8').trim();
return { inputDigest: imageFingerprint(root), imageId: actual.Id, tag: `mosaic-poc-agent:${pkg.dependencies['@earendil-works/pi-coding-agent']}-r${release}`, baseDigest: base(root), platform: 'linux/amd64' };
return { inputDigest: imageFingerprint(root), imageId: actual.Id, tag: `mosaic-poc-agent:${locked}-r${release}`, baseDigest: base(root), platform: 'linux/amd64' };
}
test('image receipt records the actual pinned FROM rather than a stale mutable tag', t => {
@@ -43,14 +44,15 @@ test('image receipt records the actual pinned FROM rather than a stale mutable t
const calls = [];
const invoke = args => {
calls.push(args);
if (args[0] === 'compose') return '';
if (args[0] === 'build') return '';
if (args[2] === 'node:24-bookworm-slim') return JSON.stringify({ RepoDigests: ['node@sha256:' + 'b'.repeat(64)] });
return JSON.stringify(actual);
};
const result = prepareImage(root, { grant: 'fixture-only', tempRoot: root }, invoke);
// The fixture has no node_modules, so a stand-in takes the image fit step's place (#1873 S5).
const result = prepareImage(root, { grant: 'fixture-only', tempRoot: root }, invoke, () => {});
assert.equal(result.baseDigest, selected);
assert.equal(calls.some(args => args[2] === 'node:24-bookworm-slim'), false);
assert.equal(calls.filter(args => args[0] === 'compose').length, 1);
assert.equal(calls.filter(args => args[0] === 'build').length, 1);
});
test('a well-formed stale base receipt refuses before any image inspection', t => {
+14 -7
View File
@@ -5,6 +5,12 @@ const image=await import('../../../scripts/testing/lane-image.mjs').catch(e=>{if
const docker=await import('../../../scripts/testing/lane-docker.mjs').catch(e=>{if(e.code!=='ERR_MODULE_NOT_FOUND')throw e;return {};});
function fixtureBase(root){return /^FROM (\S+)$/m.exec(readFileSync(join(root,'Containerfile'),'utf8'))[1];}
// #1873 S5: prepareImage runs the image fit step before the build; these fixtures have no node_modules,
// so they pass a stand-in step that records that it ran.
const fitStandIn=calls=>checkout=>calls.push(['fit',checkout]);
// The tag the fixture's copied lock gives (RELEASE 0.0.1), read here from the lock, so these tests
// follow the lock when it resolves a newer pi.
const lockTagOf=root=>'mosaic-poc-agent:'+JSON.parse(readFileSync(join(root,'package-lock.json'),'utf8')).packages['node_modules/@earendil-works/pi-coding-agent'].version+'-r0.0.1';
const processingInputs=['packages/processing-control/src/state.mjs','packages/processing-control/src/entrypoint.mjs'];
function copyProcessingInputs(root){
mkdirSync(join(root,'packages/processing-control/src'),{recursive:true});
@@ -18,7 +24,7 @@ test('a changed COPY input or actual image ID refuses compatible image sharing',
copyProcessingInputs(root);
writeFileSync(join(root,'RELEASE'),'0.0.1\n');
for(const d of ['contracts','src','adapters']){mkdirSync(join(root,d));writeFileSync(join(root,d,'one'),'same');}
const digest=image.imageFingerprint(root),id='sha256:'+'a'.repeat(64);const receipt={inputDigest:digest,imageId:id,tag:'mosaic-poc-agent:0.85.1-r0.0.1',baseDigest:fixtureBase(root),platform:'linux/amd64'};
const digest=image.imageFingerprint(root),id='sha256:'+'a'.repeat(64);const receipt={inputDigest:digest,imageId:id,tag:lockTagOf(root),baseDigest:fixtureBase(root),platform:'linux/amd64'};
const inspect=()=>({Id:id,Os:'linux',Architecture:'amd64'});
assert.equal(image.verifyImage(receipt,root,inspect).imageId,id);
assert.throws(()=>image.verifyImage(receipt,root,()=>({...inspect(),Id:'sha256:'+'c'.repeat(64)})),/image ID/);
@@ -36,9 +42,10 @@ test('exclusive image preparation records actual image ID and build wall time an
for(const f of ['Containerfile','package.json','package-lock.json'])cpSync(new URL('../../../'+f,import.meta.url),join(root,f));
copyProcessingInputs(root);
writeFileSync(join(root,'RELEASE'),'0.0.1\n');for(const d of ['contracts','src','adapters'])mkdirSync(join(root,d));
const calls=[];const invoke=(args)=>{calls.push(args);if(args[0]==='compose')return '';return JSON.stringify({Id:'sha256:'+'a'.repeat(64),Os:'linux',Architecture:'amd64'});};
const calls=[];const invoke=(args)=>{calls.push(args);if(args[0]==='build')return '';return JSON.stringify({Id:'sha256:'+'a'.repeat(64),Os:'linux',Architecture:'amd64'});};
assert.throws(()=>image.prepareImage(root,{grant:'',tempRoot:root},invoke),/grant/);
const result=image.prepareImage(root,{grant:'fixture',tempRoot:root},invoke);assert.equal(result.imageId,'sha256:'+'a'.repeat(64));assert.equal(result.builds,1);assert.ok(result.buildWallMs>=0);assert.ok(calls.some(x=>x.join(' ')==='compose build'));
const result=image.prepareImage(root,{grant:'fixture',tempRoot:root},invoke,fitStandIn(calls));assert.equal(result.imageId,'sha256:'+'a'.repeat(64));assert.equal(result.builds,1);assert.ok(result.buildWallMs>=0);assert.ok(calls.some(x=>x.join(' ')==='build -f Containerfile -t '+lockTagOf(root)+' .'),'a plain docker build of the Containerfile under the lock tag (compose.yaml has no build section)');
const fitAt=calls.findIndex(x=>x[0]==='fit');assert.ok(fitAt>=0&&fitAt<calls.findIndex(x=>x[0]==='build'),'the image fit step runs, before the build');
});
test('cleanup refuses a foreign Docker label and removes only an exact owned project',()=>{
const project='mg197-a-abcdef123456',id='123456789abc';let removed=false,label='foreign';
@@ -52,7 +59,7 @@ function imageFixture(t){
for(const f of ['Containerfile','package.json','package-lock.json'])cpSync(new URL('../../../'+f,import.meta.url),join(root,f));
copyProcessingInputs(root);
writeFileSync(join(root,'RELEASE'),'0.0.1\n');for(const d of ['contracts','src','adapters'])mkdirSync(join(root,d));
const receipt={inputDigest:image.imageFingerprint(root),imageId:'sha256:'+'a'.repeat(64),tag:'mosaic-poc-agent:0.85.1-r0.0.1',baseDigest:fixtureBase(root),platform:'linux/amd64'};
const receipt={inputDigest:image.imageFingerprint(root),imageId:'sha256:'+'a'.repeat(64),tag:lockTagOf(root),baseDigest:fixtureBase(root),platform:'linux/amd64'};
return {root,receipt};
}
test('missing receipt and missing image refuse without permitting a gate',t=>{
@@ -73,12 +80,12 @@ test('a tag changed between supervised steps fails the next boundary and suppres
});
test('pinned-base rebuilds with identical files have distinct image IDs; cold and warm counters stay separate',t=>{
const {root}=imageFixture(t);let builds=0,id='sha256:'+'a'.repeat(64);
const fake=args=>{if(args[0]==='compose'){builds++;return 'fixture build';}return JSON.stringify({Id:id,Os:'linux',Architecture:'amd64'});};
const cold=image.prepareImage(root,{grant:'fixture-cold',tempRoot:root},fake);
const fake=args=>{if(args[0]==='build'){builds++;return 'fixture build';}return JSON.stringify({Id:id,Os:'linux',Architecture:'amd64'});};
const cold=image.prepareImage(root,{grant:'fixture-cold',tempRoot:root},fake,fitStandIn([]));
assert.equal(cold.builds,1);assert.ok(cold.buildWallMs>=0);assert.equal(builds,1);
for(let i=0;i<3;i++)image.verifyImage(cold,root,()=>({Id:id,Os:'linux',Architecture:'amd64'}));
assert.equal(builds,1,'warm verification builds zero images');
id='sha256:'+'c'.repeat(64);const rebuilt=image.prepareImage(root,{grant:'fixture-rebuild',tempRoot:root},fake);
id='sha256:'+'c'.repeat(64);const rebuilt=image.prepareImage(root,{grant:'fixture-rebuild',tempRoot:root},fake,fitStandIn([]));
assert.equal(rebuilt.inputDigest,cold.inputDigest);assert.equal(rebuilt.builds,1);assert.equal(builds,2);
assert.throws(()=>image.verifyImage(cold,root,()=>({Id:rebuilt.imageId,Os:'linux',Architecture:'amd64'})),/image ID/);
});
@@ -0,0 +1,496 @@
// #1873 harness watch, slice W2: the bus host's timer over the watch (src/harness-watch-host.mjs)
// and its two state files. A fake clock and stand-in watches; one run goes through the real watch
// with stand-in harnesses, and one through startHost.
import test from "node:test";
import assert from "node:assert/strict";
import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, statSync, utimesSync, writeFileSync } from "node:fs";
import { homedir, tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { DAY_MS, STAT_MS, leakChecker, startHarnessWatch, watchFile, watchLog } from "../src/harness-watch-host.mjs";
import { bootConfig, loadSystem } from "../src/config.mjs";
import { startHost } from "../src/host.mjs";
import { makeDeployment } from "../../discord/tests/helpers.mjs";
import { fixture, tmp } from "./helpers.mjs";
import { requestWatch } from "../../plugins/src/watch-request.mjs";
import { pluginPaths } from "../../plugins/src/pinned.mjs";
import * as claude from "../../plugins/harness/claude-code/index.mjs";
function scratch(t) {
const root = mkdtempSync(join(tmpdir(), "mosaic-hw-"));
t.after(() => rmSync(root, { recursive: true, force: true }));
return root;
}
// A clock the test moves: fire() runs the interval's callback, as the timer would every statMs.
function fakeClock(start = Date.parse("2026-11-20T00:00:00Z")) {
const c = { at: start, fn: null, ms: null, cleared: [], handle: { fake: true } };
return Object.assign(c, {
now: () => c.at,
setInterval: (fn, ms) => { c.fn = fn; c.ms = ms; return c.handle; },
clearInterval: (h) => c.cleared.push(h),
fire: () => c.fn(),
});
}
const record = (over = {}) => ({ harness: "claude-code", state: "fit", via: "inspect", found: true, source: "MOSAIC_CLAUDE_BIN", version: "1.0.0", digest: "a".repeat(64), lastDigest: "a".repeat(64), fit: "pass", definite: true, failed: [], line: "ok", surfaces: { help: "usage" }, checkedAt: "2026-11-20T00:00:00.000Z", changedAt: "2026-11-20T00:00:00.000Z", ...over });
const transition = (over = {}) => ({ harness: "claude-code", at: "2026-11-20T00:00:00.000Z", from: null, to: { state: "fit", version: "1.0.0", digest: "a".repeat(64) }, failed: [], line: "ok", ...over });
// A stand-in watch that counts its calls and answers what the test sets next.
function stubWatch(answer = () => ({ records: [record()], transitions: [], refused: [], gone: [] })) {
const calls = [];
const watch = async (opts) => { calls.push(opts); return answer(opts, calls.length); };
return { calls, watch };
}
function start(t, over = {}) {
const dataRoot = over.dataRoot ?? scratch(t);
const clock = over.clock ?? fakeClock();
const logs = [];
let key = "k1";
let status = { halted: false };
const keyCalls = [];
const w = over.stub ?? stubWatch();
const handle = startHarnessWatch({
dataRoot, clock, log: (l) => logs.push(l), env: over.env ?? {}, watch: w.watch,
keys: async () => { keyCalls.push(key); return key; },
processing: () => status,
scratchRoot: () => over.scratchRoot ?? scratch(t),
});
t.after(() => handle.stop());
return { dataRoot, clock, logs, handle, w, keyCalls, setKey: (k) => { key = k; }, setStatus: (s) => { status = s; } };
}
async function tick(s) {
s.clock.fire();
await s.handle.settled();
}
test("W2 host watch: one full pass at start; a stat pass with no change runs none; a changed key runs one; a day runs one", async (t) => {
const s = start(t);
await s.handle.settled();
assert.equal(s.clock.ms, STAT_MS);
assert.equal(s.w.calls.length, 1, "the start pass");
assert.ok(existsSync(watchFile(s.dataRoot)));
s.clock.at += STAT_MS;
await tick(s);
assert.deepEqual([s.w.calls.length, s.keyCalls.length], [1, 2], "a stat pass reads the key and runs no full pass");
s.setKey("k2");
s.clock.at += STAT_MS;
await tick(s);
assert.equal(s.w.calls.length, 2, "a changed key runs one full pass");
s.clock.at += STAT_MS;
await tick(s);
assert.equal(s.w.calls.length, 2, "and only one");
s.clock.at += DAY_MS - 2 * STAT_MS;
await tick(s);
assert.equal(s.w.calls.length, 2, "less than a day since the last full pass");
s.clock.at += 2 * STAT_MS;
await tick(s);
assert.equal(s.w.calls.length, 3, "the daily pass");
// The watch gets the last records, the clock, a scratch directory and the stop signal.
const opts = s.w.calls[2];
assert.equal(opts.last[0].harness, "claude-code");
assert.equal(opts.now().getTime(), s.clock.at);
assert.equal(typeof opts.scratch, "string");
assert.equal(existsSync(opts.scratch), false, "its scratch directory is removed after the pass");
assert.equal(opts.signal.aborted, false);
});
test("W2 host watch: while processing is halted or invalid no pass runs; the first tick after resume runs one full pass", async (t) => {
const s = start(t);
await s.handle.settled();
s.setStatus({ halted: true });
for (let i = 0; i < 3; i++) { s.clock.at += STAT_MS; await tick(s); }
assert.deepEqual([s.w.calls.length, s.keyCalls.length], [1, 1], "halted: no key read, no pass");
s.setStatus({ halted: null, invalid: true, code: "processing-state-invalid" });
s.clock.at += STAT_MS;
await tick(s);
assert.equal(s.w.calls.length, 1, "invalid: no pass");
assert.equal(s.logs.filter((l) => /halted or invalid/.test(l)).length, 1, "logged once while held");
s.setStatus({ halted: false });
s.clock.at += STAT_MS;
await tick(s);
assert.equal(s.w.calls.length, 2, "resume runs a full pass though the key did not change");
s.clock.at += STAT_MS;
await tick(s);
assert.equal(s.w.calls.length, 2, "once");
});
test("W2 host watch: a processing state that can't be read holds the start pass", async (t) => {
const dataRoot = scratch(t);
const clock = fakeClock();
const w = stubWatch();
const h = startHarnessWatch({ dataRoot, clock, watch: w.watch, keys: async () => "k", processing: () => { throw new Error("unreadable"); }, scratchRoot: () => scratch(t) });
t.after(() => h.stop());
await h.settled();
assert.equal(w.calls.length, 0);
});
test("W2 host watch: passes never overlap; a tick while one runs does nothing", async (t) => {
let release;
const stub = stubWatch((opts, n) => n === 1 ? new Promise((done) => { release = () => done({ records: [record()], transitions: [], refused: [], gone: [] }); }) : { records: [record()], transitions: [], refused: [], gone: [] });
const s = start(t, { stub });
await new Promise((done) => setImmediate(done));
assert.equal(s.w.calls.length, 1);
const running = s.handle.settled();
for (let i = 0; i < 5; i++) { s.setKey(`k${i + 2}`); s.clock.at += STAT_MS; s.clock.fire(); }
await new Promise((done) => setImmediate(done));
assert.deepEqual([s.w.calls.length, s.keyCalls.length], [1, 1], "no second pass and no key read while the first runs");
release();
await running;
s.clock.at += STAT_MS;
await tick(s);
assert.equal(s.w.calls.length, 2, "the next tick after it runs normally");
});
test("W2 host watch: a pass that throws is logged with no path and the timer goes on", async (t) => {
const dir = scratch(t);
const stub = stubWatch((opts, n) => { if (n === 1) throw Object.assign(new Error(`boom at ${dir}/secret`), {}); if (n === 2) throw Object.assign(new Error(`eacces ${dir}`), { code: "EACCES" }); return { records: [record()], transitions: [], refused: [], gone: [] }; });
const s = start(t, { stub });
await s.handle.settled();
assert.equal(s.logs.length, 1);
assert.match(s.logs[0], /harness watch: a pass failed \(an error\); the timer goes on/);
assert.ok(!s.logs[0].includes(dir));
assert.equal(existsSync(watchFile(s.dataRoot)), false);
s.clock.at += STAT_MS;
await tick(s);
assert.equal(s.w.calls.length, 2, "the key was not kept, so the next tick runs a full pass again");
assert.match(s.logs[1], /\(EACCES\)/);
s.clock.at += STAT_MS;
await tick(s);
assert.equal(s.w.calls.length, 3);
assert.ok(existsSync(watchFile(s.dataRoot)));
assert.deepEqual(s.clock.cleared, [], "the timer was never cleared");
// A keys read that throws is a failed pass too.
const k = startHarnessWatch({ dataRoot: scratch(t), clock: fakeClock(), log: (l) => s.logs.push(l), watch: stubWatch().watch, keys: async () => { throw new Error(dir); }, processing: () => ({ halted: false }), scratchRoot: () => scratch(t) });
t.after(() => k.stop());
await k.settled();
assert.match(s.logs.at(-1), /a pass failed \(an error\)/);
});
test("W2 host watch: stop clears the timer, aborts the pass running, waits for it, and writes nothing after", async (t) => {
let seen = null;
let ended = false;
const stub = stubWatch((opts) => new Promise((done) => {
seen = opts.signal;
opts.signal.addEventListener("abort", () => setTimeout(() => { ended = true; done({ records: [record()], transitions: [transition()], refused: [], gone: [], stopped: true }); }, 50));
}));
const s = start(t, { stub });
await new Promise((done) => setImmediate(done));
assert.equal(seen.aborted, false);
await s.handle.stop();
assert.equal(seen.aborted, true);
assert.equal(ended, true, "stop waited for the pass to end");
assert.deepEqual(s.clock.cleared, [s.clock.handle]);
assert.equal(existsSync(watchFile(s.dataRoot)), false);
assert.equal(existsSync(watchLog(s.dataRoot)), false);
s.clock.fire();
await s.handle.settled();
assert.equal(s.w.calls.length, 1, "a tick after stop does nothing");
assert.deepEqual(readdirSync(stub.calls[0].scratch.replace(/\/[^/]+$/, "")), [], "the pass's scratch directory is gone");
// A watch that ignores the signal and answers normally still writes nothing once stopped.
const late = start(t, { stub: stubWatch((opts) => new Promise((done) => opts.signal.addEventListener("abort", () => done({ records: [record()], transitions: [transition()], refused: [], gone: [] })))) });
await new Promise((done) => setImmediate(done));
await late.handle.stop();
assert.equal(existsSync(watchFile(late.dataRoot)), false);
assert.deepEqual(late.logs, []);
});
test("W2 host watch: the json is replaced whole and holds no surfaces; the jsonl only grows, one line per transition", async (t) => {
let answer = { records: [record()], transitions: [transition()], refused: [{ event: "plugin-refused", kind: "harness", name: "odd", code: "plugin-interface", message: "x" }], gone: [] };
const s = start(t, { stub: stubWatch(() => answer) });
await s.handle.settled();
const file = watchFile(s.dataRoot);
const first = JSON.parse(readFileSync(file, "utf8"));
assert.deepEqual(Object.keys(first).sort(), ["checkedAt", "gone", "harnesses", "refused", "version"]);
assert.equal(first.version, 1);
assert.equal(first.checkedAt, new Date(s.clock.at).toISOString());
assert.equal(first.harnesses.length, 1);
assert.equal("surfaces" in first.harnesses[0], false, "surfaces are slice W6's");
assert.equal(first.refused[0].name, "odd");
assert.equal(statSync(file).mode & 0o777, 0o600);
assert.equal(statSync(join(s.dataRoot, "state")).mode & 0o777, 0o700);
const log = watchLog(s.dataRoot);
assert.equal(readFileSync(log, "utf8").split("\n").filter(Boolean).length, 1);
assert.equal(statSync(log).mode & 0o777, 0o600);
const ino = statSync(file).ino;
const before = readFileSync(log, "utf8");
// A second pass with two transitions and one harness gone.
answer = { records: [record({ state: "red", fit: "fail", failed: ["--strict-mcp-config"] })], transitions: [transition({ to: { state: "red", version: "1.0.0", digest: "a".repeat(64) } }), transition({ harness: "pi" })], refused: [], gone: ["pi"] };
s.setKey("k2");
s.clock.at += STAT_MS;
await tick(s);
const second = JSON.parse(readFileSync(file, "utf8"));
assert.notEqual(statSync(file).ino, ino, "renamed over, not written in place");
assert.deepEqual([second.harnesses[0].state, second.refused, second.gone], ["red", [], ["pi"]]);
const after = readFileSync(log, "utf8");
assert.ok(after.startsWith(before), "the jsonl keeps every line it had");
assert.equal(after.split("\n").filter(Boolean).length, 3);
assert.deepEqual(after.split("\n").filter(Boolean).map((l) => JSON.parse(l).harness), ["claude-code", "claude-code", "pi"]);
// A pass with no transition appends nothing.
answer = { records: [record()], transitions: [], refused: [], gone: [] };
s.setKey("k3");
s.clock.at += STAT_MS;
await tick(s);
assert.equal(readFileSync(log, "utf8"), after);
assert.deepEqual(readdirSync(join(s.dataRoot, "state")).sort(), ["harness-watch.json", "harness-watch.jsonl"], "no temp file is left");
// A new host reads the records back as the last ones.
const next = start(t, { dataRoot: s.dataRoot });
await next.handle.settled();
assert.equal(next.w.calls[0].last[0].harness, "claude-code");
});
test("W2 host watch: a record or transition holding a host path or an env value is not written, and the log names no path", async (t) => {
const secret = "fixture-secret-value-71";
const cases = [
(o) => ({ records: [record({ line: `found at ${o.scratch}/claude` })], transitions: [], refused: [], gone: [] }),
() => ({ records: [record({ version: `1.0 ${homedir()}` })], transitions: [], refused: [], gone: [] }),
() => ({ records: [record()], transitions: [transition({ line: `token ${secret}` })], refused: [], gone: [] }),
(o) => ({ records: [record({ failed: [o.dataRoot] })], transitions: [], refused: [], gone: [] }),
];
for (const answer of cases) {
const s = start(t, { env: { FIXTURE_TOKEN: secret }, stub: stubWatch(answer) });
await s.handle.settled();
assert.equal(existsSync(watchFile(s.dataRoot)), false);
assert.equal(existsSync(watchLog(s.dataRoot)), false);
assert.match(s.logs.join("\n"), /a state line held a host path or an env value/);
for (const l of s.logs) assert.ok(!l.includes(s.dataRoot) && !l.includes(secret));
}
// An env value that is also a harness name or a state word does not block the write.
const named = start(t, { env: { MOSAIC_HARNESS: "claude-code", OTHER: "unknown-ish" } });
await named.handle.settled();
assert.ok(existsSync(watchFile(named.dataRoot)));
});
test("W2 host watch through the real watch: stand-in harnesses that print a scratch path and an env value leave neither in either file", async (t) => {
const root = scratch(t);
const dataRoot = join(root, "org");
mkdirSync(dataRoot);
const secret = "fixture-secret-value-93";
const bin = join(root, "bin", "claude");
mkdirSync(join(root, "bin"));
// Its --version prints its own path and the secret; its --help lacks every adapter flag.
writeFileSync(bin, `#!/bin/sh\ncase "$1" in\n--version) echo "1.0.0 $0 ${secret} $HOME";;\n--help) echo "usage: claude [--version] at $0";;\n*) exit 3;;\nesac\n`, { mode: 0o755 });
const env = { MOSAIC_CLAUDE_BIN: bin, MOSAIC_PI_BIN: join(root, "no-pi"), PATH: "/usr/bin:/bin", FIXTURE_TOKEN: secret, HOME: join(root, "home") };
const clock = fakeClock();
const logs = [];
const h = startHarnessWatch({ dataRoot, env, clock, log: (l) => logs.push(l), processing: () => ({ halted: false }), scratchRoot: () => root });
t.after(() => h.stop());
await h.settled();
assert.deepEqual(logs, []);
const json = readFileSync(watchFile(dataRoot), "utf8");
const jsonl = readFileSync(watchLog(dataRoot), "utf8");
const doc = JSON.parse(json);
const rec = doc.harnesses.find((r) => r.source === "MOSAIC_CLAUDE_BIN");
assert.equal(rec.state, "red");
assert.ok(rec.failed.includes("--strict-mcp-config"));
assert.ok(doc.harnesses.some((r) => r.state === "missing" && r.source === "MOSAIC_PI_BIN"));
assert.equal(jsonl.split("\n").filter(Boolean).length, doc.harnesses.length, "the first pass is a transition per harness");
for (const text of [json, jsonl]) {
for (const needle of [root, secret, homedir(), dataRoot, bin, tmpdir()]) assert.ok(!text.includes(needle), `no ${needle === secret ? "env value" : "host path"} in a state file`);
}
// Its scratch directories are gone.
assert.deepEqual(readdirSync(root).sort(), ["bin", "org"]);
// A stat pass with nothing changed runs no full pass: the jsonl and json stay as they are.
clock.at += STAT_MS;
clock.fire();
await h.settled();
assert.equal(readFileSync(watchFile(dataRoot), "utf8"), json);
// The binary replaced: its key changed, so the next tick runs a full pass.
writeFileSync(bin, `#!/bin/sh\ncase "$1" in\n--version) echo "1.0.1";;\n--help) echo "usage";;\n*) exit 3;;\nesac\n`, { mode: 0o755 });
clock.at += STAT_MS;
clock.fire();
await h.settled();
assert.notEqual(readFileSync(watchFile(dataRoot), "utf8"), json, "a changed key ran a full pass");
assert.equal(JSON.parse(readFileSync(watchFile(dataRoot), "utf8")).harnesses.find((r) => r.source === "MOSAIC_CLAUDE_BIN").version, "1.0.1");
});
// Review P3-6: each needle of the backstop on its own: the data root, the home directory (with HOME
// unset, so no env value covers it) and a value only the host process's env holds.
test("W2 host watch: the backstop knows the data root, the home directory and the host's own env values", () => {
const leaks = leakChecker({ env: {}, scratch: "/fixture-scratch-x", dataRoot: "/fixture-data-root/org" });
assert.equal(leaks({ line: "at /fixture-data-root/org/state" }), true);
assert.equal(leaks({ line: "at /fixture-scratch-x/home" }), true);
assert.equal(leaks({ line: "nothing here" }), false);
assert.equal(leaks({ harness: "/fixture-data-root/org" }), false, "a name field is not text");
const saved = process.env.HOME;
process.env.FIXTURE_HW_ONLY = "fixture-process-only-77";
try {
delete process.env.HOME;
const home = homedir();
const bare = leakChecker({ env: {}, scratch: "/fixture-scratch-x", dataRoot: "/fixture-data-root/org" });
assert.equal(bare({ version: `1.0 ${home}` }), true, "the home directory");
assert.equal(bare({ line: "token fixture-process-only-77" }), true, "a value of the host process's env");
} finally {
process.env.HOME = saved;
delete process.env.FIXTURE_HW_ONLY;
}
});
// Review P3-5: a file: URL, a host:/path and a ~/ path a binary prints are paths too, and reach
// neither file.
test("W2 host watch through the real watch: a file: URL, a host:/path and a ~/ path in a version reach neither file", async (t) => {
const root = scratch(t);
const dataRoot = join(root, "org");
mkdirSync(dataRoot);
const bin = join(root, "bin", "claude");
mkdirSync(join(root, "bin"));
writeFileSync(bin, `#!/bin/sh\ncase "$1" in\n--version) echo "1.0 file:///opt/fixture-tree/bin/claude host:/srv/fixture-a/b ~/.local/share/fixture/claude";;\n--help) echo "usage";;\n*) exit 3;;\nesac\n`, { mode: 0o755 });
const env = { MOSAIC_CLAUDE_BIN: bin, MOSAIC_PI_BIN: join(root, "no-pi"), PATH: "/usr/bin:/bin", HOME: join(root, "home") };
const h = startHarnessWatch({ dataRoot, env, clock: fakeClock(), log: () => {}, processing: () => ({ halted: false }), scratchRoot: () => root });
t.after(() => h.stop());
await h.settled();
const json = readFileSync(watchFile(dataRoot), "utf8");
const jsonl = readFileSync(watchLog(dataRoot), "utf8");
assert.equal(JSON.parse(json).harnesses.find((r) => r.source === "MOSAIC_CLAUDE_BIN").version, "1.0 <path> host:<path> <path>");
for (const text of [json, jsonl]) for (const needle of ["fixture-tree", "/srv/", "fixture-a", "~/", "/fixture/claude"]) assert.ok(!text.includes(needle), `no ${needle} in a state file`);
});
// Review P2-4: a module asks for a stat pass now (claude-code's preflight, with no verdict). It runs
// at once, or once after the pass running, and a full pass only when the key changed.
test("W2 host watch: a request runs a stat pass now, a full pass only on a changed key, once after a running pass, none after stop", async (t) => {
let release = null;
let hold = false;
const stub = stubWatch(() => (hold ? new Promise((done) => { release = () => done({ records: [record()], transitions: [], refused: [], gone: [] }); }) : { records: [record()], transitions: [], refused: [], gone: [] }));
const s = start(t, { stub });
await s.handle.settled();
assert.deepEqual([stub.calls.length, s.keyCalls.length], [1, 1]);
requestWatch();
await s.handle.settled();
assert.deepEqual([stub.calls.length, s.keyCalls.length], [1, 2], "a request with the same key runs a stat pass and no full pass");
s.setKey("k2");
requestWatch();
await s.handle.settled();
assert.equal(stub.calls.length, 2, "a request after the key changed runs a full pass");
// Requests while a pass runs: one more stat pass right after it, however many came.
hold = true;
s.setKey("k3");
requestWatch();
await new Promise((done) => setImmediate(done));
assert.equal(stub.calls.length, 3);
hold = false;
s.setKey("k4");
requestWatch();
requestWatch();
assert.equal(s.keyCalls.length, 4, "no stat pass while a pass runs");
release();
await new Promise((done) => setTimeout(done, 20));
await s.handle.settled();
assert.deepEqual([stub.calls.length, s.keyCalls.length], [4, 5], "one stat pass after it, which saw k4 and ran a full pass");
await s.handle.stop();
requestWatch();
await s.handle.settled();
assert.deepEqual([stub.calls.length, s.keyCalls.length], [4, 5], "after stop a request does nothing");
});
// Review P2-4, through the real watch: a red claude touched after its check has a new stat key, so
// preflight has no verdict for a moment; it asks the watch, whose pass checks it again, and preflight
// refuses again within that pass, not at the next tick.
test("W2 host watch through the real watch: preflight's request re-checks a red claude touched after its check", async (t) => {
const root = scratch(t);
const dataRoot = join(root, "org");
mkdirSync(dataRoot);
const bin = join(root, "bin", "claude");
mkdirSync(join(root, "bin"));
writeFileSync(bin, `#!/bin/sh\ncase "$1" in\n--version) echo "1.0.0";;\n--help) echo "usage: claude [--version]";;\n*) exit 3;;\nesac\n`, { mode: 0o755 });
const env = { MOSAIC_CLAUDE_BIN: bin, MOSAIC_PI_BIN: join(root, "no-pi"), PATH: "/usr/bin:/bin", HOME: join(root, "home") };
const launch = { ...env, ANTHROPIC_API_KEY: "fixture-api-key" };
const clock = fakeClock();
const h = startHarnessWatch({ dataRoot, env, clock, log: () => {}, processing: () => ({ halted: false }), scratchRoot: () => root });
t.after(() => h.stop());
await h.settled();
assert.equal(claude.preflight({ env: launch })?.code, "harness-feature-missing");
chmodSync(bin, 0o700);
utimesSync(bin, new Date(), new Date(Date.now() + 60000));
assert.equal(claude.preflight({ env: launch }), null, "no verdict for the new stat key: this launch is not held");
await h.settled();
assert.equal(claude.preflight({ env: launch })?.code, "harness-feature-missing", "the requested pass checked it again, with no tick");
});
test("W2 startHost with harnessWatch starts the watch in the host, and close stops it", async (t) => {
const root = tmp(t);
const f = fixture(root);
makeDeployment(root);
const boot = bootConfig({ system: loadSystem({ env: f.env }), businessId: "acme", env: f.env });
const clock = fakeClock();
let signal = null;
const stub = stubWatch((opts) => new Promise((done) => { signal = opts.signal; opts.signal.addEventListener("abort", () => done({ records: [], transitions: [], refused: [], gone: [], stopped: true })); }));
const host = await startHost({ boot, business: "acme", log: () => {}, harnessWatch: { env: {}, clock, watch: stub.watch, keys: async () => "k", processing: () => ({ halted: false }), scratchRoot: () => scratch(t) } });
let closed = false;
t.after(() => (closed ? null : host.close(0)));
await new Promise((done) => setTimeout(done, 50));
assert.equal(stub.calls.length, 1, "the start pass runs in the host");
assert.equal(stub.calls[0].dataRoot, boot.dataRoot);
assert.equal(signal.aborted, false);
assert.equal(await host.close(0), 0);
closed = true;
assert.equal(signal.aborted, true, "close aborted the pass");
assert.deepEqual(clock.cleared, [clock.handle], "and cleared the timer");
});
test("W3 host watch: harness plugins load from pluginRoot, the state files stay under the organization's dataRoot, and the backstop knows both", async (t) => {
const dataRoot = scratch(t);
const pluginRoot = scratch(t);
const clock = fakeClock();
const stub = stubWatch();
const handle = startHarnessWatch({ dataRoot, pluginRoot, clock, env: {}, watch: stub.watch, keys: async () => "k", processing: () => ({ halted: false }), scratchRoot: () => scratch(t) });
t.after(() => handle.stop());
await handle.settled();
assert.equal(stub.calls.length, 1);
assert.equal(stub.calls[0].dataRoot, pluginRoot, "the watch loads harness plugins from the plugin root");
assert.ok(existsSync(watchFile(dataRoot)), "the state file is under the organization's dataRoot");
assert.equal(existsSync(watchFile(pluginRoot)), false, "and not under the plugin root");
assert.equal(leakChecker({ env: {}, scratch: null, dataRoot, pluginRoot })({ line: `${pluginRoot}/plugins` }), true, "a record naming the plugin root is not written");
assert.throws(() => startHarnessWatch({ dataRoot, pluginRoot: "", watch: stub.watch }), TypeError);
});
test("W3 startHost gives the watch the instance root as its plugin root when the organization has one", async (t) => {
const root = tmp(t);
const f = fixture(root);
makeDeployment(root);
const boot = bootConfig({ system: loadSystem({ env: f.env }), businessId: "acme", env: f.env });
const instanceRoot = scratch(t);
const clock = fakeClock();
const stub = stubWatch();
const host = await startHost({ boot: { ...boot, instanceRoot }, business: "acme", log: () => {}, harnessWatch: { env: {}, clock, watch: stub.watch, keys: async () => "k", processing: () => ({ halted: false }), scratchRoot: () => scratch(t) } });
let closed = false;
t.after(() => (closed ? null : host.close(0)));
for (let n = 0; n < 250 && !existsSync(watchFile(boot.dataRoot)); n++) await new Promise((done) => setTimeout(done, 20));
assert.equal(stub.calls.length, 1);
assert.equal(stub.calls[0].dataRoot, instanceRoot, "plugins load from the instance root, as the launcher loads them");
assert.ok(existsSync(watchFile(boot.dataRoot)), "the state file is under the organization's root");
assert.equal(existsSync(watchFile(instanceRoot)), false);
assert.equal(await host.close(0), 0);
closed = true;
});
test("W3 host watch: the default stat key reads the plugin root's plugins, and a record naming the plugin root is not written", async (t) => {
const dataRoot = scratch(t);
// The plugin root sits beside TMPDIR, not in it: TMPDIR is an env value the backstop already
// knows, so a root under it would be caught without the plugin root as a needle of its own.
// With TMPDIR unset, its value cannot mask the check, so a separate scratch directory works.
// Otherwise skip if the parent cannot take a sibling: a child would give a false pass.
let pluginRoot;
try { pluginRoot = process.env.TMPDIR ? mkdtempSync(join(dirname(tmpdir()), "mosaic-hw-plugins-")) : scratch(t); }
catch { t.skip("the plugin root must sit beside TMPDIR, and TMPDIR's parent is not writable"); return; }
t.after(() => rmSync(pluginRoot, { recursive: true, force: true }));
const clock = fakeClock();
const logs = [];
let line = "ok";
const stub = stubWatch(() => ({ records: [record({ line })], transitions: [], refused: [], gone: [] }));
const handle = startHarnessWatch({ dataRoot, pluginRoot, clock, env: {}, log: (l) => logs.push(l), watch: stub.watch, processing: () => ({ halted: false }), scratchRoot: () => scratch(t) });
t.after(() => handle.stop());
await handle.settled();
assert.equal(stub.calls.length, 1);
clock.fire(); await handle.settled();
assert.equal(stub.calls.length, 1, "no change, no full pass");
// A plugin lock that can't be read changes the plugin root's key (the loader refuses it): a full pass.
writeFileSync(pluginPaths(pluginRoot).lock, "{", { mode: 0o600 });
line = `${pluginRoot}/plugins/pi`;
assert.equal(leakChecker({ env: {}, scratch: null, dataRoot })({ line }), false, "without the plugin root the backstop does not catch this path");
clock.fire(); await handle.settled();
assert.equal(stub.calls.length, 2, "the key read the plugin root, so its change ran a full pass");
assert.match(logs.join("\n"), /a state line held a host path or an env value/, "and the record that names the plugin root was not written");
assert.ok(!logs.join("\n").includes(pluginRoot));
assert.ok(!readFileSync(watchFile(dataRoot), "utf8").includes(pluginRoot));
});
@@ -77,6 +77,7 @@ async function start(t, { f, adapter }, threadIdle) {
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
threadsState: (m) => runtime.threadsState(m.business),
threadIdle: (m) => threadIdle(runtime, m),
failureNotice: (m) => runtime.failureNotice(m.record),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
@@ -95,6 +95,7 @@ async function start(t, { f, adapter }, { tickMs = 100, ops: replace } = {}) {
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
threadsState: (m) => runtime.threadsState(m.business),
threadIdle: (m) => runtime.threadIdle(m.record),
failureNotice: (m) => runtime.failureNotice(m.record),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
@@ -48,6 +48,7 @@ function requestHost(x) {
landHeld: m => x.bus.landHeld(m.record), landHolders: m => x.bus.landHolders(m.business),
sessionClears: m => x.bus.sessionClears(m.business),
threadsState: m => x.bus.threadsState(m.business), lifecycleRequests: m => x.bus.lifecycleRequests(m.business),
failureNotice: m => x.bus.failureNotice(m.record),
'request.admission': m => x.bus.admission(m), 'request.launches': m => x.bus.requestLaunches(m.business),
'request.launchResult': m => x.bus.requestLaunchResult(m.record), 'request.manifest': m => x.bus.requestManifest(m.record),
'request.release': m => x.bus.requestRelease(m.cap, m.record), 'request.chief': m => x.bus.requestChief(m.business),
+147 -3
View File
@@ -12,7 +12,8 @@ import { createHash } from "node:crypto";
import { once } from "node:events";
import { createInterface } from "node:readline";
import { connect } from "node:net";
import { chmodSync, existsSync, mkdirSync, readFileSync, readdirSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs";
import { chmodSync, existsSync, mkdirSync, readFileSync, readdirSync, renameSync, rmSync, statSync, symlinkSync, watch, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { Client } from "../../bus/src/client.mjs";
@@ -24,6 +25,8 @@ import { bootConfig, loadSystem } from "../src/config.mjs";
import { cmdlineOf, startHost, startTimeOf } from "../src/host.mjs";
import { createLauncher } from "../src/launcher.mjs";
import { mockAnthropic, REPO } from "../../harness/tests/helpers.mjs";
import { fixtureDir, piStandIn } from "../../harness/tests/pi-stand-in.mjs";
import * as pi from "../../plugins/harness/pi/index.mjs";
import { fakeHarnesses, fixture, HOST_CREDENTIALS, tmp, withoutHostCredentials } from "./helpers.mjs";
const FAKE = fileURLToPath(new URL("../../harness/tests/fixtures/fake-adapter.mjs", import.meta.url));
@@ -79,7 +82,9 @@ function commit(dir, text) {
// Every credential variable of the shell running the tests is dropped, pi's own provider keys
// included (row 332 R2); a pi chief gets one made-up pi key instead.
function prepare(t, more = (doc) => doc) {
const root = tmp(t, "mosaic-launch-");
// pi-fit-tmp re-check: its pi stand-ins sit under root, which must not be under /tmp.
const root = fixtureDir("mosaic-launch-");
t.after(() => rmSync(root, { recursive: true, force: true }));
const f = fixture(root, "acme", (doc) => more(edit(doc)));
withoutHostCredentials(f.env, { piKey: f.doc.roles.chief.vars.harness === "pi" || f.doc.roles.chief.vars["fallback.harness"] === "pi" });
f.env.CLAUDE_CODE_OAUTH_TOKEN = "stand-in";
@@ -151,7 +156,8 @@ const log = (f) => readFileSync(launchLogFile(f.dataRoot, "acme"), "utf8").trim(
test('P3 proxy key reaches the spawned runner but never session.json or the bundle', { skip, timeout: 30000 }, async t => {
const proxy = { baseUrl: 'http://127.0.0.1:12345', key: 'fx332-agent-a' };
const prepared = prepare(t, doc => {
doc.launch.models['claude-opus-5-5'] = { provider: 'proxy', model: 'claude-fixture', route: 'anthropic' };
// #1873 harness-matrix: a Claude route on pi is refused, so the key check rides a non-Claude proxy route.
doc.launch.models['claude-opus-5-5'] = { provider: 'proxy', model: 'fixture-route-model', route: 'glm' };
return doc;
});
const { launcher, f, close } = await setup(t, doc => doc, prepared, { testProxy: proxy });
@@ -265,6 +271,144 @@ test("bus start --pm: the launcher instance (the chief of staff) runs under its
assert.equal(end.exitCode, 0);
});
// #1873 pi-unpin S1/S2: the version label comes from the pi the module finds. A pi whose --version
// fails gives a null label, and the launch still goes ahead.
test("pi-unpin: a pi named by MOSAIC_PI_BIN that exits 1 on --version records a null harnessVersion and still launches", { skip, timeout: 30000 }, async (t) => {
const prepared = prepare(t);
const pkg = join(prepared.root, "pi-prefix", "lib", "node_modules", "@earendil-works", "pi-coding-agent");
mkdirSync(join(pkg, "dist", "bundle"), { recursive: true });
writeFileSync(join(pkg, "package.json"), JSON.stringify({ name: "@earendil-works/pi-coding-agent", version: "3.0.0", bin: { pi: "dist/bundle/cli.js" } }));
// #1873 S3: the stand-in passes the fit check; only its --version fails.
writeFileSync(join(pkg, "dist", "bundle", "cli.js"), piStandIn({ versionExit: 1 }), { mode: 0o755 });
mkdirSync(join(prepared.root, "pi-prefix", "bin"));
const bin = join(prepared.root, "pi-prefix", "bin", "pi");
symlinkSync("../lib/node_modules/@earendil-works/pi-coding-agent/dist/bundle/cli.js", bin);
prepared.f.env.MOSAIC_PI_BIN = bin;
// env() reads the bus's own environment, as the launcher's is process.env outside tests.
const before = process.env.MOSAIC_PI_BIN;
process.env.MOSAIC_PI_BIN = bin;
t.after(() => { if (before === undefined) delete process.env.MOSAIC_PI_BIN; else process.env.MOSAIC_PI_BIN = before; });
const { f, launcher, close } = await setup(t, (doc) => doc, prepared, { harnessVersion: undefined });
const chief = await launcher.launchPm();
assert.equal(chief.harness, "pi");
const [s] = readSessions(f.dataRoot);
const manifest = JSON.parse(readFileSync(join(s.runDir, "bundle", "manifest.json"), "utf8"));
assert.equal(manifest.harnessVersion, null);
assert.equal(await close(), 0);
});
// #1873 re-check 8: a pi stand-in under root/pi-prefix, as an npm prefix holds it. Returns its bin.
function piPrefix(root, knobs) {
const pkg = join(root, "pi-prefix", "lib", "node_modules", "@earendil-works", "pi-coding-agent");
mkdirSync(join(pkg, "dist", "bundle"), { recursive: true });
writeFileSync(join(pkg, "package.json"), JSON.stringify({ name: "@earendil-works/pi-coding-agent", version: "3.0.0", bin: { pi: "dist/bundle/cli.js" } }));
writeFileSync(join(pkg, "dist", "bundle", "cli.js"), piStandIn(knobs), { mode: 0o755 });
mkdirSync(join(root, "pi-prefix", "bin"));
const bin = join(root, "pi-prefix", "bin", "pi");
symlinkSync("../lib/node_modules/@earendil-works/pi-coding-agent/dist/bundle/cli.js", bin);
return bin;
}
// Each fit run makes one scratch directory under tmpdir(), so a watch there counts the runs.
// quiet() waits until no run has started for a while, so a check an earlier test left going
// (the real pi's, warmed by its launcher) is not counted.
function fitRuns(t) {
const made = new Set();
const watcher = watch(tmpdir(), (event, name) => { if (name && name.startsWith("mosaic-pi-fit-")) made.add(name); });
t.after(() => watcher.close());
const quiet = async () => {
let n;
do { n = made.size; await new Promise((r) => setTimeout(r, 700)); } while (made.size !== n);
};
return { made, quiet };
}
function busPiBin(t, prepared, bin) {
prepared.f.env.MOSAIC_PI_BIN = bin;
const before = process.env.MOSAIC_PI_BIN;
process.env.MOSAIC_PI_BIN = bin;
t.after(() => { if (before === undefined) delete process.env.MOSAIC_PI_BIN; else process.env.MOSAIC_PI_BIN = before; });
}
// #1873 pi-unpin re-check 8: the launcher starts pi's fit check when it starts, before any launch,
// and does not wait for it. The first launch joins that check: the two together make one check's
// runs, not two.
test("pi-unpin re-check 8: the launcher starts pi's fit check when it starts, and the first launch joins it", { skip, timeout: 60000 }, async (t) => {
const { made, quiet } = fitRuns(t);
// One check on its own, for another stand-in: how many runs it makes.
const aloneRoot = fixtureDir("mosaic-launch-pi-");
t.after(() => rmSync(aloneRoot, { recursive: true, force: true }));
const alone = piPrefix(aloneRoot, {});
await quiet();
made.clear();
assert.equal(await pi.preflight({ env: { ANTHROPIC_API_KEY: "stand-in", MOSAIC_PI_BIN: alone } }), null);
await quiet();
const runs = made.size;
assert.ok(runs > 10, `one check made ${runs} runs`);
const prepared = prepare(t);
busPiBin(t, prepared, piPrefix(prepared.root, {}));
made.clear();
const { launcher, close } = await setup(t, (doc) => doc, prepared, { harnessVersion: undefined });
// No launch yet, and the check is under way.
await until(() => made.size > 0, 10000).catch(() => assert.fail("no fit run started before the first launch"));
const chief = await launcher.launchPm();
assert.equal(chief.harness, "pi");
await quiet();
assert.equal(made.size, runs, "the launch joined the check the launcher started");
assert.equal(await close(), 0);
});
// A warm-up that fails leaves no unhandled rejection and keeps no verdict: the next preflight runs
// the check again and refuses on its own.
test("pi-unpin re-check 8: a fit check the launcher starts that fails is dropped: no unhandled rejection, nothing kept", { skip, timeout: 60000 }, async (t) => {
const { made, quiet } = fitRuns(t);
const unhandled = [];
const seen = (e) => unhandled.push(e);
process.on("unhandledRejection", seen);
t.after(() => process.off("unhandledRejection", seen));
const prepared = prepare(t);
const bin = piPrefix(prepared.root, { dropFlags: ["--no-themes"] });
busPiBin(t, prepared, bin);
await quiet();
made.clear();
const { close } = await setup(t, (doc) => doc, prepared, { harnessVersion: undefined });
await until(() => made.size > 0, 10000).catch(() => assert.fail("no fit run started before the first launch"));
// This preflight joins the failing check; then it is gone.
const env = { ANTHROPIC_API_KEY: "stand-in", MOSAIC_PI_BIN: bin };
const refusal = { code: "harness-feature-missing", message: "pi 9.9.9 (MOSAIC_PI_BIN) lacks --no-themes" };
assert.deepEqual(await pi.preflight({ env }), refusal);
await quiet();
await new Promise((r) => setImmediate(r));
assert.deepEqual(unhandled, []);
made.clear();
assert.deepEqual(await pi.preflight({ env }), refusal);
await quiet();
assert.ok(made.size > 0, "the failed check was kept");
assert.equal(await close(), 0);
});
// Re-check 9: a harness plugin the warm-up can't load leaves one launch log entry, by its code
// only, so the message (which may name a host path) stays out of it.
test("pi-unpin re-check 9: a harness plugin the launcher's warm-up can't load leaves one launch log entry, by code only", { skip, timeout: 30000 }, async (t) => {
const given = prepare(t, (doc) => {
doc.plugins = { ...doc.plugins, harness: [...doc.plugins.harness, "broken"] };
return doc;
});
const makeLoader = await fakeHarnesses(t, { pi: given.adapter, "claude-code": given.adapter }, (write) => {
write("harness", "broken", `throw new Error("broken at /srv/fit-secret-path");\n`);
});
const { f, close } = await setup(t, undefined, given, { makeLoader });
const refused = () => existsSync(launchLogFile(f.dataRoot, "acme")) ? log(f).filter((e) => e.event === "plugin-refused") : [];
await until(() => refused().length > 0, 10000).catch(() => assert.fail("the warm-up left no record of the refused plugin"));
await new Promise((r) => setTimeout(r, 500));
const entries = refused();
assert.equal(entries.length, 1, JSON.stringify(entries));
const [entry] = entries;
assert.deepEqual(Object.keys(entry).sort(), ["at", "code", "during", "event", "kind", "name"]);
assert.deepEqual([entry.kind, entry.name, entry.during], ["harness", "broken", "warm-up"]);
assert.match(entry.code, /^[a-z][a-z-]+$/);
assert.doesNotMatch(readFileSync(launchLogFile(f.dataRoot, "acme"), "utf8"), /fit-secret-path/);
assert.equal(await close(), 0);
});
test("the chief of staff launches a coder through its launch tool; the coder answers; refusals name their code", { skip, timeout: 30000 }, async (t) => {
const { f, launcher, client, cto } = await setup(t);
const sock = launchSocketPath(f.dataRoot);
+1
View File
@@ -95,6 +95,7 @@ async function start(t, { f, adapter }, options = {}) {
definitionScan: (m) => runtime.definitionScan({ business: m.business, definition: m.definition, entry: m.entry }),
threadsState: (m) => runtime.threadsState(m.business),
threadIdle: (m) => runtime.threadIdle(m.record),
failureNotice: (m) => runtime.failureNotice(m.record),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
@@ -21,12 +21,20 @@ const id = value => createHash('sha256').update(value).digest('hex').slice(0,12)
// #774 R2: extra names more harnesses to switch on and load (a fallback route's); real names the
// harnesses loaded as the real module, so their own preflight runs; hostEnv adds launcher env
// such as MOSAIC_CLAUDE_BIN.
// #1873 harness-matrix: an OAuth Claude model runs on claude-code only, so the pi stream case reads
// a non-Claude model the proxy serves in Anthropic format (upstream other) from a stand-in catalogue.
// The chief's model key stays claude-opus-5-5; on pi its upstream model is not a Claude name.
export const PI_CHIEF_MODEL='fixture-chief-model';
const STREAM_OTHER=Object.freeze({provider:'other',model:'fixture-stream-model',module:'cliproxy',upstream:'other'});
const streamCatalogue=()=>({invalidate(){},read:async()=>({status:'ready',observedAt:'2026-11-19T00:00:00.000Z',notices:[],
models:[{key:'other-fixture-stream-model',module:'cliproxy',upstream:'other',provider:'other',model:'fixture-stream-model',label:'other / fixture-stream-model',available:true,reason:'unverified',harnesses:['claude-code','pi']}]})});
export async function managedLaunchFixture(t,{proxy=true,launches=true,refuseProbe=false,harness='pi',bwrap=null,admit=null,extra=[],real=[],hostEnv={},modelStream=null,modelCatalogue=null}={}){
const root=fs.mkdtempSync(join(tmpdir(),'ml653-')),closers=[],history=new Set(['fx653-manage']),proxyReceipts=[],logs=[];
let server,bus,launcher,stderr;const trace={preflightAttempts:0,successfulPreflights:0,enableBeforeSpawn:[],bindings:[],stalls:[],ops:[]};
const checkHistory=entries=>{const text=JSON.stringify(entries);for(const value of history)if(text.includes(value))throw Error('fixture credential leaked');};
t.after(async()=>{const failures=[];for(const close of [...closers].reverse()){try{await close();}catch{failures.push('owned-close');}}try{server?.closeAllConnections();if(server?.listening)await new Promise((resolve,reject)=>server.close(e=>e?reject(e):resolve()));}catch{failures.push('owned-server-close');}try{checkHistory({logs,proxyReceipts,trace});}catch{failures.push('value-free-receipts');}if(stderr)process.stderr.write=stderr;try{fs.rmSync(root,{recursive:true,force:true});}catch{failures.push('owned-root-close');}if(failures.length)throw new AggregateError(failures.map(code=>Error(code)),'owned managed fixture cleanup failed');});
const f=fixture(root,'acme',doc=>{doc.plugins={...doc.plugins,harness:[harness,...extra]};for(const role of Object.values(doc.roles))role.vars={...role.vars,harness};doc.roles.chief.vars.model='claude-opus-5-5';if(modelStream){doc.launch.models['claude-opus-5-5']={provider:'anthropic',model:'claude-opus-5-5'};}return doc;});
const f=fixture(root,'acme',doc=>{doc.plugins={...doc.plugins,harness:[harness,...extra]};for(const role of Object.values(doc.roles))role.vars={...role.vars,harness};doc.roles.chief.vars.model='claude-opus-5-5';if(harness==='pi')doc.launch.models['claude-opus-5-5']={provider:'zai',model:PI_CHIEF_MODEL};if(modelStream){doc.launch.models['claude-opus-5-5']=harness==='pi'?STREAM_OTHER:{provider:'anthropic',model:'claude-opus-5-5'};if(harness==='pi')doc.plugins={...doc.plugins,'model-provider':['cliproxy']};}return doc;});
if(modelStream&&harness==='pi'&&!modelCatalogue)modelCatalogue=streamCatalogue();
const env={PATH:process.env.PATH,HOME:process.env.HOME,TMPDIR:process.env.TMPDIR,MOSAIC_CONFIG:f.config,MOSAIC_ROLES_DIR:f.roles,...hostEnv};
// The nonproxy positive fixture supplies its own fake pi login. Managed
// proxy fixtures deliberately hold no upstream credential at all.
@@ -90,6 +98,7 @@ export async function managedLaunchFixture(t,{proxy=true,launches=true,refusePro
case 'landRelease':return bus.landRelease(m.record);
case 'threadsState':return bus.threadsState(m.business);
case 'threadIdle':return bus.threadIdle(m.record);
case 'failureNotice':return bus.failureNotice(m.record);
case 'lifecycleRequests':return bus.lifecycleRequests(m.business);
case 'sessionClears':return bus.sessionClears(m.business);
default:throw Error('fixture unrequested trusted operation');
@@ -11,8 +11,11 @@ async function launched(t,options){const f=await managedLaunchFixture(t,options)
test('653 actual managed proxy launch stays idle unarmed then stops first silent turn through trusted runtime',{skip,timeout:30000},async t=>{
const {f,s}=await launched(t);assert.equal(f.trace.successfulPreflights,1);assert.deepEqual(f.trace.enableBeforeSpawn,[true]);assert.equal(f.trace.bindings[0].modelKeyId,f.expectedKeyId());assert.equal(s.modelKeyId,f.expectedKeyId());assert.notEqual(s.pid,s.runnerPid);
await pause(1250);assert.equal(f.trace.stalls.length,0,'idle beyond configured initial budget does not stop');assert.equal(fs.existsSync(join(s.runDir,'proxy-first-turn')),false);assert.equal(startTimeOf(s.runnerPid),s.runnerStartTime);
const before=process.hrtime.bigint();f.send('SILENT');await waitFor(()=>fs.existsSync(join(s.runDir,'proxy-first-turn')),'actual managed adapter turn publishes marker');const stamp=BigInt(fs.readFileSync(join(s.runDir,'proxy-first-turn'),'utf8').trim().split(' ')[1]);assert.ok(stamp>=before&&stamp<=process.hrtime.bigint());assert.equal(fs.existsSync(join(s.runDir,'proxy-first-output')),false);
await waitFor(()=>f.trace.stalls.length===1,'actual caller watcher reaches trusted modelStall');assert.deepEqual(f.trace.stalls,[{run:s.run,keyId:s.modelKeyId,stopped:true}]);await waitFor(()=>f.sessions().length===0,'normal managed exit unregisters session');assert.equal(f.launchLog().some(r=>r.action==='proxy-stall'&&r.keyIds?.includes(s.modelKeyId)&&!r.code),true);f.assertHistoryGuard();f.valueFree();
const before=process.hrtime.bigint();const sent=f.send('SILENT');await waitFor(()=>fs.existsSync(join(s.runDir,'proxy-first-turn')),'actual managed adapter turn publishes marker');const stamp=BigInt(fs.readFileSync(join(s.runDir,'proxy-first-turn'),'utf8').trim().split(' ')[1]);assert.ok(stamp>=before&&stamp<=process.hrtime.bigint());assert.equal(fs.existsSync(join(s.runDir,'proxy-first-output')),false);
await waitFor(()=>f.trace.stalls.length===1,'actual caller watcher reaches trusted modelStall');assert.deepEqual(f.trace.stalls,[{run:s.run,keyId:s.modelKeyId,stopped:true}]);await waitFor(()=>f.sessions().length===0,'normal managed exit unregisters session');assert.equal(f.launchLog().some(r=>r.action==='proxy-stall'&&r.keyIds?.includes(s.modelKeyId)&&!r.code),true);f.assertHistoryGuard();
// #1873 chat-fail F2: the stopped turn's message gets one plain line, as a failed turn's.
const line=await f.reply(sent);assert.equal(line.body,"The turn failed: the model service gave no answer in time, so the turn was stopped. Try again, or check the account in Settings > Model setup.\nDetails: stopped: proxy-stall");assert.deepEqual([line.from_role,line.from_run,line.class],['chief','bus','RESULT']);
f.valueFree();
});
test('653 actual managed proxy output cancels caller watcher and first timestamp never resets',{skip,timeout:30000},async t=>{
const {f,s}=await launched(t);const first=f.send('OUTPUT');const answer=await f.reply(first);assert.equal(answer.body,'fx653-private-output');const file=join(s.runDir,'proxy-first-turn'),stamp=fs.readFileSync(file,'utf8');assert.equal(fs.statSync(file).mode&0o777,0o600);assert.equal(fs.readFileSync(join(s.runDir,'proxy-first-output'),'utf8'),'output\n');
@@ -90,6 +90,8 @@ async function routed(t, harness) {
const makeLoader = await fakeHarnesses(t, { pi: null, 'claude-code': null }, write => {
// fsthrows: its preflight reads a missing file, so the error's message quotes the path.
write('harness', 'fsthrows', `export * from ${PI};\nimport { readFileSync } from "node:fs";\nexport function preflight() { readFileSync(${JSON.stringify(absent)}); }\n`);
// fsrejects (#1873 re-check 8): an async preflight, as pi's is, whose promise rejects with that error.
write('harness', 'fsrejects', `export * from ${PI};\nimport { readFileSync } from "node:fs";\nexport async function preflight() { readFileSync(${JSON.stringify(absent)}); }\n`);
// plainthrows: a plain Error whose message quotes a host path, as a module's own text.
write('harness', 'plainthrows', `export * from ${PI};\nexport function preflight() { throw new Error(${JSON.stringify(`cannot read ${absent}`)}); }\n`);
// pathfails: a preflight that returns a failure whose message quotes a host path.
@@ -146,6 +148,15 @@ test('a harness whose preflight throws a system error refuses harness-failed nam
clean(JSON.stringify(s.last()), s.root);
});
// #1873 re-check 8 P3-1: a preflight promise that rejects is refused as a throw is, by the call and
// the system code, never the raw error or its path.
test('a harness whose async preflight rejects with a system error refuses harness-failed naming the call and code, never the path', async t => {
const s = await routed(t, 'fsrejects');
assert.deepEqual(await s.launch(), { ok: false, error: 'harness-failed' });
assert.equal(s.last().why, 'harness fsrejects preflight: cannot open a host file (ENOENT)');
clean(JSON.stringify(s.last()), s.root);
});
test('a harness the loader refuses names its code only, never the loader message or a path', async t => {
const s = await routed(t, 'absent');
assert.deepEqual(await s.launch(), { ok: false, error: 'harness-refused' });
+300
View File
@@ -0,0 +1,300 @@
// #1873 pi-unpin S5: package.json gives pi a range, the image tag comes from the version
// package-lock.json resolves, an image fit step runs the fit check on the locked pi before the
// image build, and the host launcher records pi's version without comparing it to anything.
// The image-building scripts (build.sh, release.sh, verify.sh, test-task.sh, test-auth.sh) need
// Docker or an instance config, so their tag and build logic is tested here through the shell
// functions and modules they call, against a stand-in lock.
import test from "node:test";
import assert from "node:assert/strict";
import { chmodSync, cpSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync, existsSync } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { tmpdir } from "node:os";
import { spawnSync } from "node:child_process";
import { fileURLToPath } from "node:url";
import { sandboxProblem } from "../../seat/src/sandbox.mjs";
import { fixtureDir, piStandIn } from "../../harness/tests/pi-stand-in.mjs";
const REPO = resolve(fileURLToPath(new URL("../../../", import.meta.url)));
const missing = (e) => { if (e.code !== "ERR_MODULE_NOT_FOUND") throw e; return {}; };
const tagMod = await import("../../../scripts/image-tag.mjs").catch(missing);
const fitMod = await import("../../../scripts/pi-fit-image.mjs").catch(missing);
const image = await import("../../../scripts/testing/lane-image.mjs").catch(missing);
const SKIP = sandboxProblem() ? { skip: "no working bwrap here" } : {};
const NAME = "@earendil-works/pi-coding-agent";
const KEY = `node_modules/${NAME}`;
const RANGE = ">=0.85.1";
const NODE_BIN = dirname(process.execPath);
const PATH = `${NODE_BIN}:/usr/bin:/bin`;
const entry = (version) => ({ version, resolved: `https://registry.npmjs.org/${NAME}/-/pi-coding-agent-${version}.tgz`, integrity: `sha512-fixture${version}` });
const lockOf = (version) => ({ name: "fixture", lockfileVersion: 3, requires: true, packages: { "": { name: "fixture", dependencies: { [NAME]: RANGE } }, ...(version ? { [KEY]: entry(version) } : {}) } });
// A checkout with package.json's range, a lock that resolves `version`, and RELEASE.
function checkout(t, { version = "9.9.9", release = "0.0.13" } = {}) {
const root = fixtureDir("s5-");
t.after(() => rmSync(root, { recursive: true, force: true }));
writeFileSync(join(root, "package.json"), JSON.stringify({ name: "fixture", dependencies: { [NAME]: RANGE } }));
writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(version)));
writeFileSync(join(root, "RELEASE"), `${release}\n`);
return root;
}
// npm ci's effect in a checkout: the package, its .bin link and node_modules/.package-lock.json.
function install(root, knobs = {}, { version = "9.9.9", hidden = entry(version) } = {}) {
const pkg = join(root, KEY);
mkdirSync(join(pkg, "dist", "bundle"), { recursive: true });
writeFileSync(join(pkg, "package.json"), JSON.stringify({ name: NAME, version, bin: { pi: "dist/bundle/cli.js" } }));
writeFileSync(join(pkg, "dist", "bundle", "cli.js"), piStandIn({ version, ...knobs }), { mode: 0o755 });
mkdirSync(join(root, "node_modules", ".bin"), { recursive: true });
symlinkSync(`../${NAME}/dist/bundle/cli.js`, join(root, "node_modules", ".bin", "pi"));
writeFileSync(join(root, "node_modules", ".package-lock.json"), JSON.stringify({ name: "fixture", lockfileVersion: 3, packages: hidden ? { [KEY]: hidden } : {} }));
return pkg;
}
const noFit = () => { throw new Error("the fit check must not run when the locked pi can't be told"); };
test("pi-unpin S5: a stand-in lock at 9.9.9 gives mosaic-poc-agent:9.9.9-r<RELEASE>, and the package.json range never appears in the tag", t => {
assert.equal(typeof tagMod.imageTag, "function", "scripts/image-tag.mjs exports imageTag");
const root = checkout(t);
const tag = tagMod.imageTag(root);
assert.equal(tag, "mosaic-poc-agent:9.9.9-r0.0.13");
assert.ok(!tag.includes(RANGE) && !tag.includes("0.85.1"), "the tag carries the lock's version, not the range");
const cli = spawnSync(process.execPath, [join(REPO, "scripts/image-tag.mjs"), root], { encoding: "utf8", env: { PATH } });
assert.equal(cli.status, 0, cli.stderr);
assert.equal(cli.stdout, "mosaic-poc-agent:9.9.9-r0.0.13\n");
// The repository itself: its lock resolves 0.85.1 under the range.
assert.equal(JSON.parse(readFileSync(join(REPO, "package.json"), "utf8")).dependencies[NAME], RANGE);
assert.equal(tagMod.imageTag(REPO), `mosaic-poc-agent:${JSON.parse(readFileSync(join(REPO, "package-lock.json"), "utf8")).packages[KEY].version}-r${readFileSync(join(REPO, "RELEASE"), "utf8").trim()}`);
});
test("pi-unpin S5: a lock with no pi entry, a range for a version, or a bad RELEASE refuses the tag, with no path in the line", t => {
assert.equal(typeof tagMod.imageTag, "function", "scripts/image-tag.mjs exports imageTag");
const none = checkout(t, { version: null });
assert.throws(() => tagMod.imageTag(none), { message: `image tag: package-lock.json has no entry for ${NAME}` });
const ranged = checkout(t);
writeFileSync(join(ranged, "package-lock.json"), JSON.stringify({ lockfileVersion: 3, packages: { [KEY]: { ...entry("9.9.9"), version: RANGE } } }));
assert.throws(() => tagMod.imageTag(ranged), { message: `image tag: package-lock.json's ${NAME} version isn't a plain version` });
const gone = checkout(t);
rmSync(join(gone, "package-lock.json"));
assert.throws(() => tagMod.imageTag(gone), { message: "image tag: package-lock.json can't be read" });
assert.throws(() => tagMod.imageTag(checkout(t, { release: "next" })), { message: "image tag: RELEASE must be a version like 0.0.13" });
const cli = spawnSync(process.execPath, [join(REPO, "scripts/image-tag.mjs"), none], { encoding: "utf8", env: { PATH } });
assert.equal(cli.status, 1);
assert.equal(cli.stdout, "");
assert.ok(!cli.stderr.includes(none), "a refusal line carries no host path");
});
// scripts/common.sh as the image-building scripts source it, in a stand-in checkout.
function shellCheckout(t, opts) {
const root = checkout(t, opts);
mkdirSync(join(root, "scripts"));
for (const f of ["common.sh", "image-tag.mjs"]) cpSync(join(REPO, "scripts", f), join(root, "scripts", f));
return root;
}
const sh = (root, script, env = {}) => spawnSync("bash", ["-c", `source scripts/common.sh && ${script}`], { cwd: root, encoding: "utf8", env: { PATH, ...env } });
test("pi-unpin S5: load_release (build.sh, release.sh, verify.sh, test-release.sh) exports the lock's tag and refuses a lock it can't read", t => {
const root = shellCheckout(t);
const r = sh(root, 'load_release && printf "%s|%s" "$MOSAIC_RELEASE" "$MOSAIC_IMAGE_TAG"');
assert.equal(r.status, 0, r.stderr);
assert.equal(r.stdout, "0.0.13|mosaic-poc-agent:9.9.9-r0.0.13");
writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(null)));
const bad = sh(root, 'load_release && printf "reached"');
assert.notEqual(bad.status, 0);
assert.equal(bad.stdout, "", "a lock with no pi entry stops the script");
assert.match(bad.stderr, /image tag: package-lock\.json has no entry/);
});
test("pi-unpin S5: build_image runs the image fit step first, and an unfit pi never reaches the image build", t => {
const root = shellCheckout(t);
mkdirSync(join(root, "bin"));
const log = join(root, "docker.log");
writeFileSync(join(root, "bin", "docker"), `#!/bin/sh\nprintf '%s\\n' "$*" >> "${log}"\n`, { mode: 0o755 });
writeFileSync(join(root, "scripts", "pi-fit-image.mjs"), `import { appendFileSync } from "node:fs";\nif (process.env.FIT === "unfit") { process.stderr.write("pi-fit-image: pi 9.9.9 (the lock's pi) lacks --offline\\n"); process.exit(1); }\nappendFileSync(${JSON.stringify(log)}, "fit\\n");\n`);
const env = { PATH: `${join(root, "bin")}:${PATH}` };
const unfit = sh(root, "load_release && build_image; echo after", { ...env, FIT: "unfit" });
assert.notEqual(unfit.status, 0);
assert.equal(unfit.stdout, "", "the calling script stops at the fit step");
assert.match(unfit.stderr, /lacks --offline/);
assert.equal(existsSync(log), false, "docker is never called for an unfit pi");
const fit = sh(root, "load_release && build_image", env);
assert.equal(fit.status, 0, fit.stderr);
assert.equal(readFileSync(log, "utf8"), "fit\nbuild -f Containerfile -t mosaic-poc-agent:9.9.9-r0.0.13 .\n");
// Without load_release there is no tag to build under.
const untagged = spawnSync("bash", ["-c", "source scripts/common.sh && build_image"], { cwd: root, encoding: "utf8", env });
assert.notEqual(untagged.status, 0);
assert.match(untagged.stderr, /build_image needs MOSAIC_IMAGE_TAG from load_release/);
});
test("pi-unpin S5: no script builds the image tag from package.json, and every image build goes through build_image", () => {
const read = (f) => readFileSync(join(REPO, f), "utf8");
const scripts = ["scripts/common.sh", "scripts/build.sh", "scripts/release.sh", "scripts/verify.sh", "scripts/test-task.sh", "scripts/test-auth.sh", "scripts/agent-host-dev.sh", "scripts/mosaic-task.mjs", "scripts/testing/lane-image.mjs"];
for (const f of scripts) {
assert.doesNotMatch(read(f), /dependencies\s*(\?\.)?\[\s*['"]@earendil-works\/pi-coding-agent['"]\s*\]/, `${f} reads pi's version from package.json`);
}
for (const f of ["scripts/build.sh", "scripts/release.sh", "scripts/verify.sh"]) {
assert.match(read(f), /^\s*build_image\b/m, `${f} builds through build_image`);
assert.doesNotMatch(read(f), /docker (compose )?build/, `${f} builds without the image fit step`);
}
// compose.yaml can't build or pull the agent image, so a missing image refuses at compose run
// instead of being built without the fit step.
const compose = read("compose.yaml");
assert.doesNotMatch(compose, /^\s*build:/m, "compose.yaml has no build section");
assert.match(compose, /^\s*pull_policy: never$/m);
// The tag comes from scripts/image-tag.mjs, and a refusal stops the script instead of running on an empty tag.
for (const f of ["scripts/test-task.sh", "scripts/test-auth.sh"]) assert.match(read(f), /^\s*TAG="\$\(node scripts\/image-tag\.mjs\)" \|\| exit 1(\s|$)/m, `${f} takes the tag from scripts/image-tag.mjs and stops when it can't`);
assert.match(read("scripts/mosaic-task.mjs"), /MOSAIC_IMAGE_TAG = tag;/);
});
// The gate's image inputs, with a lock at `version` and package.json's range.
function imageCheckout(t, version = "9.9.9") {
const root = mkdtempSync("/mnt/storage/scratch/s5i-");
t.after(() => rmSync(root, { recursive: true, force: true }));
cpSync(join(REPO, "Containerfile"), join(root, "Containerfile"));
writeFileSync(join(root, "package.json"), JSON.stringify({ name: "fixture", dependencies: { [NAME]: RANGE } }));
writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(version)));
writeFileSync(join(root, "RELEASE"), "0.0.1\n");
mkdirSync(join(root, "packages/processing-control/src"), { recursive: true });
for (const f of ["state.mjs", "entrypoint.mjs"]) cpSync(join(REPO, "packages/processing-control/src", f), join(root, "packages/processing-control/src", f));
for (const d of ["contracts", "src", "adapters"]) { mkdirSync(join(root, d)); writeFileSync(join(root, d, "one"), "same"); }
return root;
}
const base = (root) => /^FROM (\S+)$/m.exec(readFileSync(join(root, "Containerfile"), "utf8"))[1];
test("pi-unpin S5: verifyImage accepts the lock-derived tag when package.json holds a range, and refuses a tag the lock doesn't give", t => {
assert.equal(typeof image.verifyImage, "function");
const root = imageCheckout(t);
const id = "sha256:" + "a".repeat(64);
const receipt = { inputDigest: image.imageFingerprint(root), imageId: id, tag: "mosaic-poc-agent:9.9.9-r0.0.1", baseDigest: base(root), platform: "linux/amd64" };
const inspect = () => ({ Id: id, Os: "linux", Architecture: "amd64" });
assert.equal(image.verifyImage(receipt, root, inspect).tag, "mosaic-poc-agent:9.9.9-r0.0.1");
assert.throws(() => image.verifyImage({ ...receipt, tag: "mosaic-poc-agent:0.85.1-r0.0.1" }, root, inspect), /image tag differs from release identity/);
assert.throws(() => image.verifyImage({ ...receipt, tag: `mosaic-poc-agent:${RANGE}-r0.0.1` }, root, inspect), /refused/);
writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(null)));
assert.throws(() => image.verifyImage({ ...receipt, inputDigest: image.imageFingerprint(root) }, root, inspect), /image tag: package-lock\.json has no entry/);
});
test("pi-unpin S5: prepareImage tags from the lock, runs the image fit step before the build, and a refusing step stops the build", t => {
assert.equal(typeof image.prepareImage, "function");
assert.equal(typeof image.imageFitStep, "function", "lane-image.mjs exports the image fit step");
const root = imageCheckout(t);
const calls = [];
const invoke = (args) => { calls.push(args.join(" ")); return args[0] === "build" ? "" : JSON.stringify({ Id: "sha256:" + "a".repeat(64), Os: "linux", Architecture: "amd64" }); };
const receipt = image.prepareImage(root, { grant: "fixture", tempRoot: root }, invoke, (c) => calls.push(`fit ${c === root}`));
assert.equal(receipt.tag, "mosaic-poc-agent:9.9.9-r0.0.1");
assert.deepEqual(calls, ["fit true", "build -f Containerfile -t mosaic-poc-agent:9.9.9-r0.0.1 .", "image inspect mosaic-poc-agent:9.9.9-r0.0.1 --format {{json .}}"]);
const stopped = [];
assert.throws(() => image.prepareImage(root, { grant: "fixture", tempRoot: root }, (a) => stopped.push(a), () => { throw new Error("unfit"); }), /unfit/);
assert.deepEqual(stopped, [], "no build after a refusing fit step");
// The default step runs the checkout's scripts/pi-fit-image.mjs and refuses with its last line.
mkdirSync(join(root, "scripts"));
writeFileSync(join(root, "scripts", "pi-fit-image.mjs"), `process.stderr.write("pi-fit-image: pi 9.9.9 (the lock's pi) lacks --offline\\n"); process.exit(1);\n`);
assert.throws(() => image.imageFitStep(root), { message: "gate-lane: refused: image fit step refused: pi-fit-image: pi 9.9.9 (the lock's pi) lacks --offline" });
assert.throws(() => image.prepareImage(root, { grant: "fixture", tempRoot: root }, (a) => stopped.push(a)), /image fit step refused/);
assert.deepEqual(stopped, []);
writeFileSync(join(root, "scripts", "pi-fit-image.mjs"), "process.exit(0);\n");
assert.equal(image.imageFitStep(root), undefined);
});
test("pi-unpin S5: the image fit step fails on a locked stand-in pi that lacks a flag, and passes one at another version that has them all", SKIP, async t => {
assert.equal(typeof fitMod.imageFit, "function", "scripts/pi-fit-image.mjs exports imageFit");
const unfit = checkout(t);
install(unfit, { dropFlags: ["--offline"] });
const r = await fitMod.imageFit(unfit);
assert.deepEqual(r, { ok: false, message: "pi 9.9.9 (the lock's pi) lacks --offline" });
const ok = checkout(t);
install(ok);
const pass = await fitMod.imageFit(ok);
assert.equal(pass.ok, true, pass.message);
assert.equal(pass.version, "9.9.9");
assert.equal(pass.locked, "9.9.9");
});
test("pi-unpin S5: the image fit step passes on the installed pi the repository's lock resolves", SKIP, async () => {
assert.equal(typeof fitMod.imageFit, "function", "scripts/pi-fit-image.mjs exports imageFit");
const locked = JSON.parse(readFileSync(join(REPO, "package-lock.json"), "utf8")).packages[KEY].version;
const r = await fitMod.imageFit(REPO);
assert.equal(r.ok, true, r.message);
assert.equal(r.locked, locked);
assert.equal(r.version, locked);
const cli = spawnSync(process.execPath, [join(REPO, "scripts/pi-fit-image.mjs")], { encoding: "utf8", env: { PATH, HOME: process.env.HOME, TMPDIR: tmpdir() }, timeout: 120000 });
assert.equal(cli.status, 0, cli.stderr);
assert.equal(cli.stdout, `pi-fit-image: pi ${locked} (the lock's pi) passed the fit check\n`);
});
test("pi-unpin S5: the image fit step can't tell, and refuses, when node_modules doesn't hold the lock's pi", async t => {
assert.equal(typeof fitMod.imageFit, "function", "scripts/pi-fit-image.mjs exports imageFit");
const lead = "pi (the lock's pi) can't be checked: ";
const npmCi = "run npm ci --ignore-scripts in the repository first";
const cases = [
["no install", () => {}, `the installed pi isn't the one package-lock.json resolves: ${npmCi}`],
["no hidden lock entry", (root) => install(root, {}, { hidden: null }), `the installed pi isn't the one package-lock.json resolves: ${npmCi}`],
["another integrity", (root) => install(root, {}, { hidden: { ...entry("9.9.9"), integrity: "sha512-other" } }), `the installed pi isn't the one package-lock.json resolves: ${npmCi}`],
["another version installed", (root) => install(root, {}, { version: "9.9.8", hidden: entry("9.9.9") }), `the installed pi's package.json doesn't carry the lock's version: ${npmCi}`],
["a lock with no pi", (root) => { install(root); writeFileSync(join(root, "package-lock.json"), JSON.stringify(lockOf(null))); }, `image tag: package-lock.json has no entry for ${NAME}`],
["a .bin/pi that isn't executable", (root) => { install(root); chmodSync(join(root, KEY, "dist", "bundle", "cli.js"), 0o644); }, `the lookup refused the lock's pi: ${npmCi}`],
["a .bin/pi in another copy of the package", (root) => {
install(root);
const other = join(root, "other", NAME);
mkdirSync(join(other, "dist", "bundle"), { recursive: true });
writeFileSync(join(other, "package.json"), JSON.stringify({ name: NAME, version: "9.9.9", bin: { pi: "dist/bundle/cli.js" } }));
writeFileSync(join(other, "dist", "bundle", "cli.js"), piStandIn({ version: "9.9.9" }), { mode: 0o755 });
rmSync(join(root, "node_modules", ".bin", "pi"));
symlinkSync(join(other, "dist", "bundle", "cli.js"), join(root, "node_modules", ".bin", "pi"));
}, `node_modules/.bin/pi isn't the lock's pi package: ${npmCi}`],
];
for (const [name, plant, problem] of cases) {
const root = checkout(t);
plant(root);
const r = await fitMod.imageFit(root, { fit: noFit });
assert.deepEqual(r, { ok: false, message: lead + problem }, name);
assert.ok(!r.message.includes(root) && !r.message.includes(realpathSync(root)), `${name}: no host path in the line`);
}
// An answer the fit check doesn't give reads as can't tell.
const root = checkout(t);
install(root);
assert.deepEqual(await fitMod.imageFit(root, { fit: async () => undefined }), { ok: false, message: `${lead}the check gave no answer` });
// The fit check gets the found pi under the step's own label, with the module's thinking levels.
let seen;
const ok = await fitMod.imageFit(root, { fit: async (pi, opts) => { seen = { pi, opts }; return { ok: true, version: "9.9.9" }; } });
assert.equal(ok.ok, true);
assert.equal(ok.locked, "9.9.9");
assert.equal(seen.pi.label, "the lock's pi");
assert.equal(seen.pi.pkg, realpathSync(join(root, KEY)));
assert.equal("locked" in seen.pi, false);
assert.ok(Array.isArray(seen.opts.levels) && seen.opts.levels.includes("high"));
});
// scripts/agent-host-dev.sh --check in a checkout with the range in package.json and a stand-in pi.
function hostCheckout(t, piScript) {
const root = fixtureDir("s5h-");
t.after(() => rmSync(root, { recursive: true, force: true }));
for (const d of ["scripts", "agents/fixture", "contracts", "user", "node_modules/.bin", "extensions/goal"]) mkdirSync(join(root, d), { recursive: true });
cpSync(join(REPO, "scripts/agent-host-dev.sh"), join(root, "scripts/agent-host-dev.sh"));
writeFileSync(join(root, "scripts/common.sh"), 'load_config() { MOSAIC_DATA_ROOT="$PWD"; MOSAIC_PROVIDER=test; MOSAIC_MODEL=test-model; MOSAIC_ADAPTER=pi; }\n');
writeFileSync(join(root, "package.json"), JSON.stringify({ dependencies: { [NAME]: RANGE } }));
for (const f of ["agents/fixture/SOUL.md", "agents/fixture/CONTEXT.md", "contracts/CONSTITUTION.md", "contracts/STANDARDS.md", "user/USER.md", "AGENTS.md", "extensions/goal/index.ts"]) writeFileSync(join(root, f), `fixture ${f}\n`);
for (const s of ["ms-tools", "ms-file-read", "ms-file-write", "ms-conductor", "ms-communications", "ms-watch", "ms-sdlc", "ms-proactive-agent", "ms-goal", "ms-unslop"]) {
mkdirSync(join(root, "skills", s), { recursive: true });
writeFileSync(join(root, "skills", s, "SKILL.md"), `fixture ${s}\n`);
}
writeFileSync(join(root, "node_modules/.bin/pi"), piScript, { mode: 0o755 });
return root;
}
const hostCheck = (root) => spawnSync("bash", [join(root, "scripts/agent-host-dev.sh"), "fixture", "--check"], { cwd: root, encoding: "utf8", env: { PATH, HOME: root }, timeout: 20000 });
test("pi-unpin S5: scripts/agent-host-dev.sh doesn't refuse pi for its version; it records the version pi prints", t => {
const root = hostCheckout(t, "#!/bin/sh\n[ \"$1\" = --version ] && { echo 9.9.9; exit 0; }\nexit 1\n");
const r = hostCheck(root);
assert.equal(r.status, 0, r.stdout + r.stderr);
assert.match(r.stdout, /fixture: native host TUI \| Pi 9\.9\.9 \| test\/test-model/);
assert.match(r.stdout, /configuration checks passed/);
// A pi whose --version fails or prints nothing still starts; the banner says so.
for (const script of ["#!/bin/sh\nexit 3\n", "#!/bin/sh\nexit 0\n"]) {
const quiet = hostCheckout(t, script);
const q = hostCheck(quiet);
assert.equal(q.status, 0, q.stdout + q.stderr);
assert.match(q.stdout, /\| Pi version unknown \|/);
}
assert.doesNotMatch(readFileSync(join(REPO, "scripts/agent-host-dev.sh"), "utf8"), /package\.json pins|\$PIN\b/);
});
+27 -7
View File
@@ -34,10 +34,11 @@ function privateArtifacts(f, session, keys) {
f.valueFree();
}
// #1873 harness-matrix: an OAuth Claude model runs on claude-code only, so pi joins the z.ai and
// codex routes on a non-Claude model, and a Claude route on pi refuses below.
const ROUTE_MODEL = 'fixture-route-model';
for (const [provider, route, generated, api] of [
['zai', null, 'mosaic-proxy-glm', 'openai-completions'],
['anthropic', null, 'anthropic', 'anthropic-messages'],
['proxy', 'anthropic', 'anthropic', 'anthropic-messages'],
['proxy', 'glm', 'mosaic-proxy-glm', 'openai-completions'],
['proxy', 'codex', 'mosaic-proxy-codex', 'openai-responses'],
]) {
@@ -45,8 +46,7 @@ for (const [provider, route, generated, api] of [
const f = await managedLaunchFixture(t);
assert.deepEqual(f.upstreamCredentialNames, [], 'managed proxy fixture has no inherited fake upstream credential');
editBusiness(f, doc => {
doc.launch.models['claude-opus-5-5'].provider = provider;
if (route) doc.launch.models['claude-opus-5-5'].route = route;
doc.launch.models['claude-opus-5-5'] = { provider, model: ROUTE_MODEL, ...(route ? { route } : {}) };
});
const answer = await f.launcher.launchPm();
const session = f.sessions().find(value => value.run === answer.run);
@@ -69,7 +69,7 @@ for (const [provider, route, generated, api] of [
assert.equal(models.providers[generated].api, api);
assert.equal(models.providers[generated].baseUrl,
other.baseUrl + (api === 'anthropic-messages' ? '' : '/v1'));
assert.equal(models.providers[generated].models.some(model => model.id === 'claude-opus-5-5'), true);
assert.equal(models.providers[generated].models.some(model => model.id === ROUTE_MODEL), true);
for (const value of Object.values(models.providers)) assert.equal(value.apiKey, '$ANTHROPIC_API_KEY');
for (const key of [own.key, other.key]) assert.equal(text.includes(key), false, 'models contain no key value');
assert.equal(fs.statSync(path).mode & 0o777, 0o600);
@@ -79,6 +79,26 @@ for (const [provider, route, generated, api] of [
});
}
// A legacy entry with no provider plugin gets no catalogue check, so a Claude-named model on a z.ai
// or codex route is treated as OAuth too (review P1-1).
for (const [provider, route, name] of [['anthropic', null], ['proxy', 'anthropic'], ['zai', null, 'claude-opus-5-5'], ['proxy', 'glm', 'claude-opus-5-5'], ['proxy', 'codex', 'claude-sonnet-5-5']]) {
test(`#1873 a stored Claude route ${provider}${route ? ':' + route : ''}${name ? ' ' + name : ''} on pi refuses model-harness-incompatible with no key acquisition or session`, { skip, timeout: 30000 }, async t => {
const f = await managedLaunchFixture(t);
editBusiness(f, doc => {
doc.launch.models['claude-opus-5-5'].provider = provider;
if (route) doc.launch.models['claude-opus-5-5'].route = route;
if (name) doc.launch.models['claude-opus-5-5'].model = name;
});
await assert.rejects(f.launcher.launchPm(), error => error.code === 'model-harness-incompatible');
assert.equal(f.trace.preflightAttempts, 0, 'refused before P2 issues a key');
assert.equal(f.trace.bindings.length, 0);
assert.equal(f.sessions().length, 0);
assert.equal(f.runDirectories().length, 0);
assert.equal(f.launchLog().some(e => e.event === 'refused' && e.code === 'model-harness-incompatible'), true);
f.valueFree();
});
}
test('P3 joined Claude harness receives the admitted P2 key only through transient auth env', { skip, timeout: 30000 }, async t => {
const f = await managedLaunchFixture(t, { harness: 'claude-code' });
assert.deepEqual(f.upstreamCredentialNames, [], 'Claude proxy fixture holds no upstream credential');
@@ -117,7 +137,7 @@ test('P3 joined unsupported provider refuses with no key acquisition, fallback,
test('row 332 R2 joined pi launch goes live when launch.models also holds an unroutable entry, which models.json leaves out', { skip, timeout: 30000 }, async t => {
const f = await managedLaunchFixture(t);
editBusiness(f, doc => {
doc.launch.models['claude-opus-5-5'].provider = 'anthropic';
doc.launch.models['claude-opus-5-5'] = { provider: 'zai', model: ROUTE_MODEL };
doc.launch.models.router = { provider: 'openrouter', model: 'x' };
});
const answer = await f.launcher.launchPm();
@@ -126,7 +146,7 @@ test('row 332 R2 joined pi launch goes live when launch.models also holds an unr
assert.equal(f.trace.successfulPreflights, 1);
assert.equal(f.runDirectories().length, 1, 'one run directory, the live one');
const { providers } = JSON.parse(fs.readFileSync(join(session.runDir, 'home', '.pi', 'agent', 'models.json'), 'utf8'));
assert.equal(providers.anthropic.models.some(model => model.id === 'claude-opus-5-5'), true);
assert.equal(providers['mosaic-proxy-glm'].models.some(model => model.id === ROUTE_MODEL), true);
assert.equal(Object.values(providers).some(value => value.models.some(model => model.id === 'x')), false, 'no openrouter entry');
privateArtifacts(f, session, [f.bus.credentials.modelRecord('acme/chief').key]);
});
+148
View File
@@ -0,0 +1,148 @@
// proxy-ride first slice (Jason 2026-11-21): `mosaic proxy ride --instance NAME | --off`, the CLI
// loader's view of a riding instance, the bus boot config, and the worker launcher's refusal line.
// The rider is the cli fixture's config; the owner is named instance "owner" under a made-up HOME.
// The real scripts/mosaic-config.mjs runs; nothing reads a real config, proxy or secret.
import test from 'node:test';
import assert from 'node:assert/strict';
import { existsSync, lstatSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { spawnSync } from 'node:child_process';
import { fileURLToPath } from 'node:url';
import { main } from '../src/cli.mjs';
import { bootConfig, loadSystem } from '../src/config.mjs';
import { proxyCommand } from '../src/model-proxy.mjs';
import { RIDE_LINES, installPaths } from '../../model-proxy/src/settings.mjs';
import { fixture, io, tmp } from './helpers.mjs';
const MANAGE = 'ride-cli-management-key';
const CONTAINER_CLI = fileURLToPath(new URL('../../processing-control/src/container-cli.mjs', import.meta.url));
// The stand-in launch step of worker-proxy-task.test.mjs; FAKE_WP=proxy answers with the proxy it got.
const HOOK = fileURLToPath(new URL('../../processing-control/tests/fixtures/worker-proxy-hook.mjs', import.meta.url));
const code = (c) => (e) => e.code === c;
function setup(t) {
const root = tmp(t, 'rcl-');
const f = fixture(root, 'acme');
f.env.HOME = root;
const ownerDir = join(root, '.config', 'mosaic-owner'), ownerData = join(root, 'od');
mkdirSync(join(ownerDir, 'secrets'), { recursive: true, mode: 0o700 }); mkdirSync(ownerData, { mode: 0o700 });
writeFileSync(join(ownerDir, 'config.json'), JSON.stringify({ configVersion: 1, environment: 'development', dataRoot: ownerData, execution: { backend: 'docker', provider: 'fixture', model: 'fixture' } }));
writeFileSync(join(ownerData, 'instance.json'), JSON.stringify({ instanceVersion: 1, name: 'owner' }));
const value = { port: 18452, binary: installPaths(ownerData).binary };
writeFileSync(join(ownerDir, 'proxy.json'), JSON.stringify(value));
writeFileSync(join(ownerDir, 'secrets', 'proxy-management.key'), MANAGE, { mode: 0o600 });
const riderDir = join(root, 'config');
return { root, f, env: f.env, riderDir, rideFile: join(riderDir, 'proxy-ride.json'), owner: { dir: ownerDir, dataRoot: ownerData, value } };
}
const ride = (s, args, env = s.env) => { const x = io(env); return main(['proxy', 'ride', ...args], x).then(() => x); };
test('ride --instance writes the file, prints the owner port, and a second identical ride holds it', async (t) => {
const s = setup(t);
let x = await ride(s, ['--instance', 'owner']);
assert.deepEqual(JSON.parse(x.out.text), { riding: 'owner', port: 18452 });
assert.equal(readFileSync(s.rideFile, 'utf8'), '{"version":1,"instance":"owner"}\n');
assert.equal(lstatSync(s.rideFile).mode & 0o777, 0o600);
assert.equal(x.out.text.includes(MANAGE) || x.err.text.includes(MANAGE), false);
x = await ride(s, ['--instance', 'owner']);
assert.deepEqual(JSON.parse(x.out.text), { riding: 'owner', port: 18452 });
// Nothing of the owner's changed.
assert.deepEqual(JSON.parse(readFileSync(join(s.owner.dir, 'proxy.json'), 'utf8')), s.owner.value);
assert.equal(existsSync(join(s.riderDir, 'proxy.json')), false);
});
test('ride and ride --off run only from a human shell', async (t) => {
const s = setup(t);
for (const args of [['--instance', 'owner'], ['--off']]) {
await assert.rejects(ride(s, args, { ...s.env, CLAUDECODE: '1' }), code('human-required'));
await assert.rejects(ride(s, [...args, '--tester']), code('human-required'));
}
assert.equal(existsSync(s.rideFile), false);
});
test('a ride the loader refuses is removed again, with the ride code and its owner line', async (t) => {
const s = setup(t);
const cases = [
[['--instance', 'nobody'], 'proxy-ride-owner-missing', () => {}],
[['--instance', 'owner'], 'proxy-ride-owner-missing', () => rmSync(join(s.owner.dir, 'proxy.json'))],
];
for (const [args, c, prep] of cases) {
prep();
await assert.rejects(ride(s, args), (e) => e.code === c && e.message === `${c}: ${RIDE_LINES[c]}` && e.exitCode === 2);
assert.equal(existsSync(s.rideFile), false, `${c}: the refused file is gone`);
}
await assert.rejects(ride(s, ['--instance', '../x']), code('proxy-ride-invalid'));
assert.equal(existsSync(s.rideFile), false);
});
test('ride refuses beside its own proxy.json, and over a ride of another instance', async (t) => {
const s = setup(t);
writeFileSync(join(s.riderDir, 'proxy.json'), '{}');
await assert.rejects(ride(s, ['--instance', 'owner']), code('proxy-ride-conflict'));
assert.equal(existsSync(s.rideFile), false);
rmSync(join(s.riderDir, 'proxy.json'));
writeFileSync(s.rideFile, '{"version":1,"instance":"other"}\n', { mode: 0o600 });
await assert.rejects(ride(s, ['--instance', 'owner']), (e) => e.code === 'proxy-usage' && e.exitCode === 4 && /already rides instance other/.test(e.message));
assert.equal(readFileSync(s.rideFile, 'utf8'), '{"version":1,"instance":"other"}\n', 'the existing ride is left as it was');
});
test('ride --off removes the file, also when the owner is gone, and says when there was none', async (t) => {
const s = setup(t);
await ride(s, ['--instance', 'owner']);
rmSync(s.owner.dir, { recursive: true });
let x = await ride(s, ['--off']);
assert.deepEqual(JSON.parse(x.out.text), { riding: null, removed: true });
assert.equal(existsSync(s.rideFile), false);
x = await ride(s, ['--off']);
assert.deepEqual(JSON.parse(x.out.text), { riding: null, removed: false });
for (const args of [[], ['--instance'], ['--off', '--instance', 'owner'], ['--on']]) await assert.rejects(ride(s, args), code('proxy-usage'));
});
test('a validate seam that answers another owner or no port removes the file', async (t) => {
const s = setup(t);
for (const answer of [{ proxy: { port: 1, owner: { instance: 'elsewhere' } } }, { proxy: { owner: { instance: 'owner' } } }, {}]) {
const x = io(s.env);
await assert.rejects(proxyCommand(['ride', '--instance', 'owner'], x, { system: () => ({}), rideValidate: () => answer }), code('proxy-ride-owner-missing'));
assert.equal(existsSync(s.rideFile), false);
}
});
test('loadSystem drops the ridden proxy unless a worker run asks; the bus boot config is the one with no proxy', async (t) => {
const s = setup(t);
const plain = bootConfig({ system: loadSystem({ env: s.env }), businessId: 'acme', env: s.env });
await ride(s, ['--instance', 'owner']);
const off = loadSystem({ env: s.env });
assert.equal(Object.hasOwn(off, 'proxy'), false, 'every caller but a worker run sees no proxy');
const on = loadSystem({ env: s.env, ride: true });
assert.deepEqual(on.proxy, { ...s.owner.value, owner: { instance: 'owner', configDir: s.owner.dir, dataRoot: s.owner.dataRoot } });
for (const system of [off, on]) {
const boot = bootConfig({ system, businessId: 'acme', env: s.env });
assert.equal(Object.hasOwn(boot, 'proxy'), false);
assert.deepEqual(boot, plain, "the rider's bus boots exactly as it did before the ride");
}
// An instance with its own proxy keeps it in the boot config, as before.
const own = bootConfig({ system: { ...off, proxy: { port: 18460, binary: '/x' } }, businessId: 'acme', env: s.env });
assert.deepEqual(own.proxy, { port: 18460, binary: '/x' });
});
test('a ride refusal reaches loadSystem and the worker launcher as its code and owner line', async (t) => {
const s = setup(t);
writeFileSync(s.rideFile, '{"version":1,"instance":"owner","path":"/x"}', { mode: 0o600 });
const line = `proxy-ride-invalid: ${RIDE_LINES['proxy-ride-invalid']}`;
assert.throws(() => loadSystem({ env: s.env }), (e) => e.exitCode === 3 && e.message === `system config problem (mosaic-config exit 2): ${line}`);
const r = spawnSync(process.execPath, [CONTAINER_CLI, '--check'], { env: s.env, encoding: 'utf8', timeout: 20000 });
assert.equal(r.status, 3, r.stderr);
assert.equal(r.stderr, `processing: ${line}\n`);
assert.equal(r.stderr.includes(s.root), false);
});
test("a worker launch through container-cli is given the owner's proxy", async (t) => {
const s = setup(t);
const launch = () => spawnSync(process.execPath, ['--import', HOOK, CONTAINER_CLI], { env: { ...s.env, FAKE_WP: 'proxy' }, encoding: 'utf8', timeout: 20000 });
let r = launch();
assert.equal(r.status, 0, r.stderr);
assert.equal(r.stdout, 'no-owner:no-proxy', 'no ride file: no proxy, as before');
await ride(s, ['--instance', 'owner']);
r = launch();
assert.equal(r.status, 0, r.stderr);
assert.equal(r.stdout, 'owner:18452');
});
@@ -95,6 +95,7 @@ function requestHost(x) {
landHeld: m => x.bus.landHeld(m.record), landHolders: m => x.bus.landHolders(m.business),
sessionClears: m => x.bus.sessionClears(m.business),
threadsState: m => x.bus.threadsState(m.business), lifecycleRequests: m => x.bus.lifecycleRequests(m.business),
failureNotice: m => x.bus.failureNotice(m.record),
'request.admission': m => x.bus.admission(m), 'request.launches': m => x.bus.requestLaunches(m.business),
'request.launchResult': m => x.bus.requestLaunchResult(m.record), 'request.manifest': m => x.bus.requestManifest(m.record),
'request.release': m => x.bus.requestRelease(m.cap, m.record), 'request.chief': m => x.bus.requestChief(m.business),
@@ -240,8 +240,8 @@ export async function lifecycleFixture(t) {
}
async function consoleBrowser(t, f) {
const b = await browser(); t.after(() => b.close());
b.on('Fetch.requestPaused', ({ requestId, request: incoming }) => {
let b = await browser(); const consolePage = b; t.after(() => consolePage.close());
consolePage.on('Fetch.requestPaused', ({ requestId, request: incoming }) => {
const url = new URL(incoming.url);
if (incoming.method === 'POST') f.recordOwner(url.pathname);
const req = httpRequest({ host: '127.0.0.1', port: f.port, path: url.pathname + url.search, method: incoming.method,
@@ -249,9 +249,9 @@ async function consoleBrowser(t, f) {
const parts = []; res.on('data', part => parts.push(part)); res.on('end', () => {
const responseHeaders = Object.entries(res.headers).filter(([name]) => !['connection','keep-alive','transfer-encoding'].includes(name))
.flatMap(([name, value]) => (Array.isArray(value) ? value : [value]).map(value => ({ name, value: String(value) })));
b.call('Fetch.fulfillRequest', { requestId, responseCode: res.statusCode, responseHeaders, body: Buffer.concat(parts).toString('base64') }).catch(() => {});
consolePage.call('Fetch.fulfillRequest', { requestId, responseCode: res.statusCode, responseHeaders, body: Buffer.concat(parts).toString('base64') }).catch(() => {});
});
}); req.on('error', () => b.call('Fetch.failRequest', { requestId, errorReason: 'Failed' }).catch(() => {})); req.end(incoming.postData);
}); req.on('error', () => consolePage.call('Fetch.failRequest', { requestId, errorReason: 'Failed' }).catch(() => {})); req.end(incoming.postData);
});
await b.call('Fetch.enable', { patterns: [{ urlPattern: ORIGIN + '/*', requestStage: 'Request' }] });
const s = await f.current(); await b.navigate(`${ORIGIN}/#/inbox/${s.selection.decision}`);
@@ -267,7 +267,10 @@ async function consoleBrowser(t, f) {
await wait('document.querySelector("#decide-status")?.textContent.startsWith("Recorded")');
},
async offline(html) {
await b.call('Fetch.disable');
// The Console can still poll during navigation. Close it before observing the downloaded file.
await b.close();
b = await browser(); const offlinePage = b; t.after(() => offlinePage.close());
external.length = 0;
await b.call('Page.addScriptToEvaluateOnNewDocument', { source: `window.__now=1000;Date.now=()=>window.__now;window.__ticks=[];window.setInterval=fn=>{window.__ticks.push(fn);return window.__ticks.length;};window.__sounds=0;window.AudioContext=class{currentTime=0;destination={};resume(){}createOscillator(){return{connect(){},frequency:{value:0},start(){window.__sounds++},stop(){}}}createGain(){return{connect(){},gain:{value:0}}}};` });
await b.call('Network.enable'); await b.call('Network.emulateNetworkConditions', { offline: true, latency: 0, downloadThroughput: 0, uploadThroughput: 0 });
b.on('Network.requestWillBeSent', ({ request }) => { if (/^https?:/.test(request.url)) external.push(request.url); });
@@ -96,6 +96,7 @@ async function start(t, { f, adapter, access }, threadsState) {
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
threadsState: (m) => threadsState(runtime, m),
threadIdle: (m) => runtime.threadIdle(m.record),
failureNotice: (m) => runtime.failureNotice(m.record),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
@@ -92,6 +92,7 @@ async function start(t, { f, adapter }, { tickMs = 100, stopTimeoutMs } = {}) {
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
threadsState: (m) => runtime.threadsState(m.business),
threadIdle: (m) => runtime.threadIdle(m.record),
failureNotice: (m) => runtime.failureNotice(m.record),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
@@ -112,6 +112,7 @@ async function start(t, { f, adapter }, { tickMs = 100, ops: replace } = {}) {
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
threadsState: (m) => runtime.threadsState(m.business),
threadIdle: (m) => runtime.threadIdle(m.record),
failureNotice: (m) => runtime.failureNotice(m.record),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
+2 -1
View File
@@ -146,7 +146,8 @@ test('real pinned install loads bus and Console; the Console starts only with --
const s=setup(t,true);
const copied=(await s.run('stack-copy.sh',['copy',s.id,'--copies',s.copies]));assert.equal(copied.status,0,copied.stderr);
const root=join(s.copies,s.id),url=f=>pathToFileURL(join(root,f)).href;
const {checkEnginePin}=await import(url('packages/conversation/src/pi-pin.mjs'));assert.equal(checkEnginePin(root).version,'0.85.1');
await import(url('packages/conversation/src/pi-seal.mjs'));const {locate}=await import(url('packages/plugins/harness/pi/binary.mjs')),{levels}=await import(url('packages/plugins/harness/pi/index.mjs')),{piFit}=await import(url('packages/harness/src/pi-fit.mjs'));
const copyPi=locate({});assert.equal(copyPi.label,"the repository's pi");assert.equal(copyPi.bin,join(root,'node_modules','.bin','pi'));const fit=await piFit(copyPi,{levels:levels()});assert.equal(fit.ok,true,fit.message);assert.equal(fit.version,'0.85.1');
const home=join(s.root,'home'),f=fixture(home),configDir=join(home,'.config','mosaic-pin');
mkdirSync(join(home,'.config'),{recursive:true});renameSync(join(home,'config'),configDir);chmodSync(configDir,0o700);
writeFileSync(join(f.dataRoot,'instance.json'),JSON.stringify({instanceVersion:1,name:'pin'}));
+3 -1
View File
@@ -2210,10 +2210,12 @@ export class Store {
// C1's scripts/mosaic runs B's host, as a copy-check failure needs.
// Row 294: config.mjs imports access/transport. Task 5b also imports the built-in catalogue and
// its task, bus, seat and processing-control dependencies; the rehearsal needs those files too.
// proxy-ride-1: config.mjs imports the ride refusal lines from model-proxy/src/settings.mjs, whose
// common.mjs reads model-proxy/PIN when it loads.
const RCOPIES=join(base,'rcopies');mkdirSync(RCOPIES);
for(const [id,v,froms] of [[B,'3g',['3f']],[C,'3h',['3f','3g']],[A,'3g',['3f']],[C1,'3g',['3f']]]){
const d=join(RCOPIES,id);
for(const p of ['scripts/mosaic','scripts/mosaic-config.mjs','packages/cli/src/config.mjs','packages/cli/src/errors.mjs','packages/cli/src/access.mjs','packages/cli/src/org-selector.mjs','packages/cli/src/transport.mjs','packages/bus/src/business.mjs','packages/bus/src/organization-context.mjs','packages/bus/src/request-store.mjs','packages/bus/src/requirement-catalog.mjs','packages/tasks/src','packages/seat/src/landing.mjs','packages/seat/src/checkout.mjs','packages/processing-control/src','packages/plugins/src/kinds.mjs','packages/organization/src','packages/business/src','packages/library/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
for(const p of ['scripts/mosaic','scripts/mosaic-config.mjs','packages/cli/src/config.mjs','packages/cli/src/errors.mjs','packages/cli/src/access.mjs','packages/cli/src/org-selector.mjs','packages/cli/src/transport.mjs','packages/bus/src/business.mjs','packages/bus/src/organization-context.mjs','packages/bus/src/request-store.mjs','packages/bus/src/requirement-catalog.mjs','packages/tasks/src','packages/seat/src/landing.mjs','packages/seat/src/checkout.mjs','packages/processing-control/src','packages/model-proxy/src','packages/model-proxy/PIN','packages/plugins/src/kinds.mjs','packages/organization/src','packages/business/src','packages/library/src','roles'])cpSync(join(repo,p),join(d,p),{recursive:true});
// Row 333 R2 (#1831): what the business loader imports from outside its directory, model-quota.mjs included.
for(const p of businessImportedFiles(repo))if(!existsSync(join(d,p)))cpSync(join(repo,p),join(d,p));
if(id===C1)writeFileSync(join(d,'scripts','mosaic'),`#!/usr/bin/env bash\nexec node "${join(RCOPIES,B,'packages','cli','src','cli.mjs')}" "$@"\n`,{mode:0o755});
+73
View File
@@ -93,6 +93,7 @@ async function start(t, { f, adapter }, { tickMs = 100, ops: replace } = {}) {
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
threadsState: (m) => runtime.threadsState(m.business),
threadIdle: (m) => runtime.threadIdle(m.record),
failureNotice: (m) => runtime.failureNotice(m.record),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
@@ -496,6 +497,78 @@ test("REQ-LIMIT-3 REQ-LIMIT-4 a thread start whose instance is at its token limi
assert.equal(launchLog(f).filter((e) => e.event === "refused" && e.thread === A).length, 1);
});
// #1873 chat-fail F2: a refused start leaves one plain line in the chat for each message it left
// unanswered, with the cause and next step, never one per retry and never the refusal's reason.
const LIMIT_LINE = "The turn failed: the agent reached a usage limit. Wait for it to reset, or check the limits in Settings.\nDetails: start refused: limit-reached";
test("#1873 F2 a refused thread start puts one plain line per unanswered message in the chat, not one per retry", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t, { edit: atLimit });
const { f } = prepared;
const host = await start(t, prepared);
const project = host.call("project.create", { name: "stack work", repo: "stack" }).id;
const A = host.call("thread.create", { project, name: "alpha" }).id;
const first = host.call("thread.send", { thread: A, body: "for A" });
const lines = (sent) => host.call("thread.messages", { thread: A }).filter((m) => m.in_reply_to === sent.id);
const [line] = await until(() => lines(first).length && lines(first));
assert.equal(line.body, LIMIT_LINE);
assert.deepEqual([line.from_role, line.from_run], ["assistant", "bus"]);
assert.equal(line.to_role, "human");
assert.equal(line.class, "RESULT");
assert.doesNotMatch(line.body, /token-limit|tokens\.instances|"limit"/, "the refusal's reason stays out of the chat");
for (let i = 0; i < 3; i++) await host.launcher.tick();
assert.equal(lines(first).length, 1, "a retry adds no line");
// A second message gets its own line; the first keeps its one.
const second = host.call("thread.send", { thread: A, body: "again" });
await until(() => lines(second).length);
for (let i = 0; i < 3; i++) await host.launcher.tick();
assert.deepEqual(lines(second).map((m) => m.body), [LIMIT_LINE]);
assert.equal(lines(first).length, 1);
assert.deepEqual(threadRuns(f, A), []);
// The messages stay pending: the start is retried and answers them once the limit allows.
assert.equal(launchLog(f).filter((e) => e.event === "refused" && e.thread === A).length, 1);
});
// #1873 review round 1 (P2-2): launch-failed can be a lost race that the next try wins, so one
// such refusal posts nothing and the answer arrives alone; the same refusal twice posts one line.
const failedLines = (host, thread) => host.call("thread.messages", { thread }).filter((m) => m.body.startsWith("The turn failed"));
test("#1873 F2 one launch-failed refusal that the next start clears leaves no failure line", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t);
let throws = 1;
const host = await start(t, prepared, { ops: (runtime) => ({ landHeld: (m) => { if (throws-- > 0) throw new Error("fixture: lost race"); return runtime.landHeld(m.record); } }) });
const project = host.call("project.create", { name: "stack work", repo: "stack" }).id;
const A = host.call("thread.create", { project, name: "alpha" }).id;
const sent = host.call("thread.send", { thread: A, body: "for A" });
const reply = await until(() => host.call("thread.messages", { thread: A }).find((m) => m.in_reply_to === sent.id && m.class === "RESULT"));
assert.match(reply.body, /THREAD \{/, "the first reply is the answer");
assert.equal(throws, -1, "the first start was refused");
assert.deepEqual(failedLines(host, A), []);
});
test("#1873 F2 the same launch-failed refusal twice for one message posts one line", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t);
const host = await start(t, prepared, { ops: () => ({ landHeld: () => { throw new Error("fixture: always refused"); } }) });
const project = host.call("project.create", { name: "stack work", repo: "stack" }).id;
const A = host.call("thread.create", { project, name: "alpha" }).id;
host.call("thread.send", { thread: A, body: "for A" });
const [line] = await until(() => failedLines(host, A).length && failedLines(host, A));
assert.match(line.body, /\nDetails: start refused: launch-failed$/);
for (let i = 0; i < 3; i++) await host.launcher.tick();
assert.equal(failedLines(host, A).length, 1);
});
// #1873 review round 1 (P3-3): a line whose post fails is posted on a later tick, once.
test("#1873 F2 a failure line whose post is refused once arrives on a later tick, exactly once", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t, { edit: atLimit });
let throws = 1;
const host = await start(t, prepared, { ops: (runtime) => ({ failureNotice: (m) => { if (throws-- > 0) throw new Error("fixture: bus busy"); return runtime.failureNotice(m.record); } }) });
const project = host.call("project.create", { name: "stack work", repo: "stack" }).id;
const A = host.call("thread.create", { project, name: "alpha" }).id;
host.call("thread.send", { thread: A, body: "for A" });
const [line] = await until(() => failedLines(host, A).length && failedLines(host, A));
assert.equal(line.body, LIMIT_LINE);
assert.equal(throws, -1, "the first post was refused");
for (let i = 0; i < 3; i++) await host.launcher.tick();
assert.equal(failedLines(host, A).length, 1);
assert.equal(host.logs.filter((l) => /failure line refused/.test(l)).length, 1);
});
test("REQ-LIMIT-4 a thread run at its token limit stops at the end of its turn: the reply lands, and the end record names the limit", { skip, timeout: 60000 }, async (t) => {
const prepared = prepare(t, { edit: atLimit });
const { f } = prepared;
@@ -97,6 +97,7 @@ async function start(t, { f, adapter, access }) {
credentialStatus: (m) => runtime.credentialStatus(m.business, m.role),
threadsState: (m) => runtime.threadsState(m.business),
threadIdle: (m) => runtime.threadIdle(m.record),
failureNotice: (m) => runtime.failureNotice(m.record),
mayLaunch: (m) => runtime.mayLaunch(m),
landHeld: (m) => runtime.landHeld(m.record),
landHolders: (m) => runtime.landHolders(m.business),
+1 -1
View File
@@ -12,7 +12,7 @@ import { join } from "node:path";
import { createHash, randomBytes } from "node:crypto";
import { run } from "../src/cli.mjs";
import { AGENT_MARKERS } from "../src/transport.mjs";
import { DEFAULT_ROTATE_GRACE_MINUTES, keyPaths, retireAtFor, rotateByFor } from "../src/triage-key.mjs";
import { DEFAULT_ROTATE_GRACE_MINUTES, keyPaths, retireAtFor, rotateByFor } from "../../bus/src/triage-key.mjs";
import { ModelKeys } from "../../bus/src/model-keys.mjs";
import { ManagementClient } from "../../model-proxy/src/client.mjs";
import { startBroker } from "../../bus/src/runtime.mjs";
+72 -15
View File
@@ -234,7 +234,7 @@ live session, and the controller never writes a session file (W11).
| `src/cohort.mjs` | `ScopeLauncher` (systemd user scope plus `shim.mjs`), `PgroupLauncher`, force stop, cohort and boot proofs |
| `src/shim.mjs` | the scope's first process, outside the `engine` cgroup; ops `hello`, `events`, `members`, `term`, `freeze`, `kill`, `release`; ignores SIGTERM and exits on `release` once `engine` is empty |
| `src/guard.mjs` | `LiveSessionGuard`: refuses live sessions and paths under live roots |
| `src/pi-pin.mjs` | the Pi pin (0.85.1 and its integrity) and the seal |
| `src/pi-seal.mjs` | the seal (with `--no-mcp` when the pi's help lists it) and the engine refusal codes |
| `src/engine.mjs`, `src/framing.mjs` | the engine link and LF framing |
| `src/turns.mjs` | `Tracker`: runs, the dispatch slot, overlap signals O1–O6 |
| `src/events.mjs` | native event to CHAT-01 event mapping |
@@ -289,16 +289,72 @@ can disagree with it.
(`--no-session`, `--fork`, `--export`, `--print`, `--continue`, ...) or a
bare word, which Pi reads as a prompt (lead decision 31; N24, including the missing flag
through `checkSeal`).
- **Engine command and environment (slice 1 S5, #1522).** The controller
always launches `node <pinRoot>/node_modules/@earendil-works/pi-coding-agent/dist/bundle/cli.js`
and checks that at construction and again at bind. `engine` takes only
- **Engine command and environment (slice 1 S5, #1522; #1873 S4).** The
controller launches `node <the real file of the pi the lookup finds>`.
The lookup is the pi harness module's (`packages/plugins/harness/pi/binary.mjs`):
`MOSAIC_PI_BIN`, else the first `pi` on `PATH`, else the repository's
copy. The `pi` option takes a lookup answer and defaults to
`locate(process.env)`. The controller checks the command at construction
and again at bind, resolving the bin's real path each time, so a bin link
moved after construction refuses `unsealed-engine`. `engine` takes only
`extraArgs`, `cwd` and `envKeys`; any other key (`command`, `preArgs`,
`env`), or an `engine` that isn't an object, refuses `unsealed-engine`.
The fake engine comes in through `opts[TEST_ENGINE]`, a symbol key that
JSON config can't carry, so no config file reaches an unsealed command.
`preArgs` carrying `--extension` still refuses there too. The binding's
`argvDigest` records the full command line.
- **Engine environment.** The engine gets `ENGINE_ENV` (`pi-pin.mjs`):
- **Engine fit and pins (#1873 S4).** No pi is refused for its version.
Before a claim, a resume or a launch, the controller runs the fit check
(`packages/harness/src/pi-fit.mjs`) on the pi the lookup finds. A pi that
lacks a feature the seal needs, or that the check can't read, refuses
`engine-feature-missing` with the fit check's line (for example
`pi 9.9.9 (MOSAIC_PI_BIN) lacks --mode rpc`). The claim's pins are
`engineVersion` (what `--version` printed, or null), `enginePin` (the fit
check's digest of the pi's files) and `argvDigest`. A resume or recovery
whose pins differ refuses `engine-pin-mismatch`, so changed bytes under a
stopped claim can't resume it. A claim made before #1873 holds npm's
integrity in `enginePin`; it refuses `engine-pin-mismatch` with "the claim
was made under the old pin record; start a new claim". Like any pin
mismatch, that refusal holds at every start of the session, since a resume
is checked against the session's last proven stop. A new session starts a
new claim.
- **Same file, same interpreter.** The fit check runs the bin as a program,
so its first line picks what runs it, while the launch runs the bin's real
file with this Node. A pi whose first line isn't `#!/usr/bin/env node` or
`#!` and an absolute path that resolves to this Node refuses
`engine-feature-missing`. So does a first line that doesn't end within the
256 bytes the kernel reads, or one with a CR before its newline. Under
`#!/usr/bin/env node`, env takes node from the fit check's PATH (the
repository's `.bin`, then this process's PATH, as `sessionEnv` builds it),
and it looks inside the fit check's sandbox, which shows only `/usr`,
`/etc`, the host's `/bin`, `/lib`, `/lib64` and `/sbin`, and this Node's
own file at its path unless that path is under `/tmp` (a session's sandbox
mounts its private `/tmp` over it, and the fit check stays undecided for a
node there). Up to the first `node`, every entry must read the
same in the sandbox as on the host: absolute (env reads an empty or
relative one against the fit check's directory), already in plain form (a
`..` after a link goes where the link points), and not in `/proc` or
`/dev`, which the sandbox makes its own, by its spelling or by the real
path of the nearest part of it that resolves. The sandbox's `/tmp` is its
own too, but starts empty, so a `/tmp` entry is judged like any other
directory the sandbox doesn't show. An entry or a `node` link there that
can't be checked refuses too. No `node` the sandbox can't show
(or reaches through a link it can't show) may come before the first one it
can, and that first one must be this Node. No `node` anywhere on the PATH
may lie inside the package the fit check mounts. Otherwise the start
refuses `engine-feature-missing`. A refusal line, the fit check's
included, names the pi by the lookup's label, which is words such as
`MOSAIC_PI_BIN`, or "the pi" for any other label. A lookup that found no
pi refuses with its own line, or "the pi lookup gave no binary" when that
line holds a path or a control character.
The controller keeps a frozen plain copy of the lookup's answer, its own
string data fields only, so an answer that changes later changes nothing.
That copy must still be the lookup's at each check: the lookup run again on
its bin gives the same `root`, `dest`, `view` and `pkg`, so the fit check
mounts, runs and digests the package the launch runs. A package link moved
since the lookup, an answer changed by hand, or a bin the lookup no longer
takes refuses `unsealed-engine`.
- **Engine environment.** The engine gets `ENGINE_ENV` (`pi-seal.mjs`):
`PATH`, `HOME`, `USER`, `LOGNAME`, `SHELL`, `LANG`, `LC_ALL`,
`LC_CTYPE`, `TZ`, `TERM`, `TMPDIR`, `PI_CODING_AGENT_DIR`, `PI_OFFLINE`,
`PI_SKIP_VERSION_CHECK` and `PI_TELEMETRY`, plus the names in
@@ -472,7 +528,7 @@ different IDs, so overlap at the seam can't be deduplicated (CHAT-01 lines
## Slash text
`textPolicy` refuses any prompt whose text, after leading whitespace, starts
with `/`: pinned Pi runs extension commands, skills (`/skill:`) and prompt
with `/`: Pi runs extension commands, skills (`/skill:`) and prompt
templates from index 0 (S1, S2). `!`, `!!` and `@` are interpreted only by
Pi's interactive mode and CLI, not on the RPC prompt path, so they are
admitted and sent as text. A `/` on a later line is not interpreted
@@ -547,13 +603,14 @@ runs in CHAT-01 `check.mjs` order, then the CHAT-03 narrowings.
| `eligibility` | launch with a used or unknown eligibility record, one from another incarnation, or a changed reservation (K17) |
| `already-active`, `unsafe-replacement`, `foreign-host` | the writer claim (W2, W3, W17) |
| `live-session-refused` | the live-session guard, at construction, bind and launch (G1–G3) |
| `unsealed-engine`, `engine-pin-mismatch` | the seal or the Pi pin (N24) |
| `unsealed-engine`, `engine-pin-mismatch` | the seal, or the engine's version, digest or argv changed under a claim (N24, W9, S4c) |
| `engine-feature-missing` | the fit check found the pi lacks a feature, or couldn't tell (S4b) |
| `configuration` | a required constructor path is missing or misplaced, the session file is unreadable, or the session header ID isn't a CHAT-01 ID |
`malformed`, `eligibility` and `preflight` are names this build adds; the
brief describes those refusals without naming them.
The controller, claim store, live-session guard and engine pin throw
The controller, claim store, live-session guard and engine checks throw
`ControlRefusal` (`safe-fs.mjs`), a subclass of the reader's `Refusal`. These
codes reach a socket client, never the reader's HTTP routes, so the control
board's status map (`REFUSAL_STATUS`, which its tests check against every
@@ -563,15 +620,15 @@ board ever serves the controller, these codes need statuses there first.
Receipt reason codes, not refusals: `transport-unknown`,
`handled-without-run`, `ack-without-start`, `interrupted`, `run-overlap`.
## Findings against pinned Pi
## Findings against Pi 0.85.1
From `tests/smoke.test.mjs`, which starts the pinned binary sealed, in a
From `tests/smoke.test.mjs`, which starts the repository's pi (0.85.1) sealed, in a
scratch home with an empty agent dir and no inherited environment, and sends
no prompt:
- It starts with no credentials and answers `get_state`, `get_commands`,
`clear_queue`, `abort` and `get_tree`. Every field the fake engine sends
for those commands, pinned Pi sends with the same type. `clear_queue` emits
for those commands, Pi 0.85.1 sends with the same type. `clear_queue` emits
`queue_update` with empty queues before its response, as the fake does.
Pi writes `auth.json` (`{}`) and `models-store.json` into the agent dir.
- **Startup append.** Pi appends to the session at startup in two cases
@@ -593,13 +650,13 @@ no prompt:
| Suite | Covers |
|---|---|
| `claim.test.mjs` | W1–W17, W20, G1–G3: the writer claim, crash barriers, the guard |
| `claim.test.mjs` | W1–W17, W20, G1–G3, S4a–S4h: the writer claim, crash barriers, the guard, the engine fit, pins and seal, the node-script check and the lookup answer check |
| `races.test.mjs` | H1–H4, H9–H23: takeover, Interrupt and force stop, retries, incarnations |
| `turns.test.mjs` | N1–N25, N24b: the turn tracker against the fake engine's Pi behaviors; the engine seal and environment |
| `turns.test.mjs` | N1–N25, N24b, N24c: the turn tracker against the fake engine's Pi behaviors; the engine seal, `--no-mcp` and environment |
| `cohort.test.mjs` | K1–K19, R1–R3, R5–R13, E1–E6: scopes, force stop, proofs, recovery, eligibility, the scope's environment, scope release and its retry after a crash, the engine-exit stop (needs a systemd user manager) |
| `flows.test.mjs` | S1–S7, P3, E1–E7, the terminal, and a CHAT-01 schema check of every record produced |
| `smoke.test.mjs` | the pinned Pi binary, as above |
| `smoke.test.mjs` | the repository's Pi binary, as above, and its fit check |
`fake-pi.mjs` models pinned Pi's RPC mode, including the startup append, and
`fake-pi.mjs` models Pi 0.85.1's RPC mode, including the startup append, and
`ctrl-child.mjs` runs a controller in a child process for the crash tests.
Fixtures live in temporary directories and are removed after each file.
+236 -20
View File
@@ -16,7 +16,7 @@
// as in CHAT-01. With no verifier nothing verifies, and every stop ends
// `uncertain`.
import { lstatSync, mkdirSync, readFileSync, unlinkSync } from "node:fs";
import { closeSync, lstatSync, mkdirSync, openSync, readFileSync, readSync, readlinkSync, realpathSync, statSync, unlinkSync } from "node:fs";
import { createServer } from "node:net";
import { basename, dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
@@ -30,7 +30,11 @@ import { LineSplitter, encodeLine, parseLine } from "./framing.mjs";
import { LiveSessionGuard, realPath } from "./guard.mjs";
import { ID, fragments, safeId } from "./parts.mjs";
import { parseSnapshot } from "./pi.mjs";
import { ENGINE_PIN_MISMATCH, PI_BIN, UNSEALED_ENGINE, argvDigest, buildPiArgs, checkEnginePin, checkSeal, engineEnv } from "./pi-pin.mjs";
import { ENGINE_FEATURE_MISSING, ENGINE_PIN_MISMATCH, OLD_PIN_LINE, UNSEALED_ENGINE, argvDigest, buildPiArgs, checkSeal, engineEnv, oldPinRecord } from "./pi-seal.mjs";
import { PACKAGE, executable, locate } from "../../plugins/harness/pi/binary.mjs";
import { sessionEnv } from "../../seat/src/session.mjs";
import { levels } from "../../plugins/harness/pi/index.mjs";
import { piFit } from "../../harness/src/pi-fit.mjs";
import { ACTOR, conversationId, createReader, rootsFromSpecs } from "./reader.mjs";
import { clone, equal, hash, newId, receiptAllows, record, scopeMatch, sealProof, sha256, targetOf } from "./records.mjs";
import { ControlRefusal, Refusal } from "./safe-fs.mjs";
@@ -79,7 +83,7 @@ const STOP_IN_PROGRESS = new Set(["fenced", "cancelling", "stopping"]);
const STOP_NEXT = { fenced: ["cancelling", "stopping", "uncertain"], cancelling: ["stopping", "uncertain"], stopping: ["uncertain"], uncertain: [] };
// check.mjs `ending`: the binding follows only stops that end the engine.
const ending = (mode) => mode === "force-stop" || mode === "engine-exit";
const SEAL_BASIS = "seal: --no-extensions and no --extension argument (pi-pin.mjs SEAL_FLAGS)";
const SEAL_BASIS = "seal: --no-extensions and no --extension argument (pi-seal.mjs SEAL_FLAGS)";
const DIALOG_REASON = "Pi dialogs are not answered in CHAT-03 (lead decision 30); shown disabled";
const refused = (code, extra = {}) => ({ outcome: `refused:${code}`, refusal: code, ...extra });
@@ -152,13 +156,74 @@ export function sessionTools(parsed) {
return tools;
}
// A first line that runs node: #!/usr/bin/env node, or #! and an absolute path
// whose last part is node, with nothing after it.
const NODE_SHEBANG = /^#![ \t]*(?:\/usr\/bin\/env[ \t]+node|(\/[^\s]*\/node))[ \t]*$/;
const ANSWER_KEYS = ["root", "dest", "view", "pkg"];
const ANSWER_FIELDS = ["bin", "label", "problem", ...ANSWER_KEYS];
// #1873 S4: what the fit check's sandbox (systemBinds in seat/src/sandbox.mjs)
// shows as the host has it: /usr, /etc and the host's /bin, /lib, /lib64 and
// /sbin, and this node's own file at process.execPath. shownInFit follows p
// one part at a time as the sandbox would and gives the file it reaches, or
// null when a link or the file lies anywhere else. Null may be too strict (the
// package's own mount is left out on purpose: no node may come from it), never
// too loose.
const SHOWN_DIRS = ["/usr", "/etc", "/bin", "/lib", "/lib64", "/sbin"];
const shownDir = (p) => SHOWN_DIRS.some((d) => p === d || p.startsWith(d + "/"));
// The sandbox makes its own /proc and /dev, so a path there names something
// else inside it than on the host. Its /tmp is another directory too, but one
// that starts empty (pi-fit.mjs makes it fresh for each run), so an entry
// there holds no node in the sandbox, and a node the host finds there counts
// as one the sandbox can't show. That holds for this Node's own file too. The
// fit sandbox now mounts that file over its fresh /tmp (pi-fit-tmp's
// mountOrder), but a session's sandbox mounts its private /tmp last and
// covers it, and the fit check stays undecided for a node under /tmp, so an
// engine whose node sits under /tmp still can't run.
const OWN_COPY_DIRS = ["/proc", "/dev"];
const underTmp = (p) => p === "/tmp" || p.startsWith("/tmp/");
function shownInFit(p, hops = 0) {
if (hops > 40) return null;
const parts = p.split("/").filter(Boolean);
let at = "/";
for (let i = 0; i < parts.length; i++) {
const next = join(at, parts[i]);
let st;
try { st = lstatSync(next); } catch { return null; }
if (st.isSymbolicLink()) {
if (!shownDir(next)) return null;
let target;
try { target = readlinkSync(next); } catch { return null; }
return shownInFit(resolve(at, target, ...parts.slice(i + 1)), hops + 1);
}
at = next;
}
return (at === process.execPath && !underTmp(at)) || shownDir(at) ? at : null;
}
// #1873 S4: the lookup's answer, copied once into a frozen plain object of
// its own data fields, so what the seal checks and what the fit check reads
// are the same values for the controller's life. A getter, a Proxy that
// answers differently later, or an inherited field changes nothing after
// this copy.
function answerCopy(pi) {
if (!pi || typeof pi !== "object") return pi;
const out = {};
for (const k of ANSWER_FIELDS) {
let d;
try { d = Object.getOwnPropertyDescriptor(pi, k); } catch { d = undefined; }
if (d && "value" in d && (typeof d.value === "string" || d.value === null)) out[k] = d.value;
}
return Object.freeze(out);
}
export class Controller {
constructor(opts = {}) {
const {
fixtureRoot, claimRoot, socketDir, sessionFile, seat, workspace = "default",
engine = {}, launcher = new PgroupLauncher(), verifier = null, host, identity, barrier = null, units = systemdUnits,
grants = null, guardOptions = {}, timeouts = {}, now = () => new Date(),
policyRevision = "policy-fixture-1", sourceRootRef = "source-fixture-1", approvedMappings = null, pinRoot = REPO_ROOT, auditWritable = true,
policyRevision = "policy-fixture-1", sourceRootRef = "source-fixture-1", approvedMappings = null, pi = locate(process.env), auditWritable = true,
hostId = null,
} = opts;
for (const [k, v] of Object.entries({ claimRoot, socketDir, sessionFile, seat })) {
@@ -180,12 +245,15 @@ export class Controller {
this.conversation = conversationId(this.root, basename(this.paths.sessionFile));
if (!engine || typeof engine !== "object" || Array.isArray(engine)) throw new ControlRefusal(UNSEALED_ENGINE, "engine is not an object");
const extra = Object.keys(engine).find((k) => !ENGINE_KEYS.has(k));
if (extra !== undefined) throw new ControlRefusal(UNSEALED_ENGINE, `engine.${extra.slice(0, 40)} can't be configured; the controller launches the pinned Pi with its own environment`);
if (extra !== undefined) throw new ControlRefusal(UNSEALED_ENGINE, `engine.${extra.slice(0, 40)} can't be configured; the controller launches the pi the lookup finds, with its own environment`);
const test = opts[TEST_ENGINE] ?? null;
// #1873 S4: the pi is binary.mjs's answer ({ bin, label, ... } or
// { bin: null, problem }). A sealed launch runs its bin's real file.
this.pi = answerCopy(pi);
this.engine = {
sealed: test === null,
command: test ? test.command : process.execPath,
preArgs: test ? test.preArgs : [join(pinRoot, PI_BIN)],
preArgs: test ? test.preArgs : [this.#piFile()],
extraArgs: engine.extraArgs ?? [],
cwd: engine.cwd ?? projectRoot,
env: test ? test.env : engineEnv(engine.envKeys),
@@ -194,8 +262,8 @@ export class Controller {
if (!Array.isArray(this.engine[k])) throw new ControlRefusal(UNSEALED_ENGINE, `engine.${k} is not a list`);
}
if (typeof this.engine.command !== "string" || !this.engine.command) throw new ControlRefusal(UNSEALED_ENGINE, "the engine command is not a path");
this.noMcp = false;
this.piArgs = buildPiArgs({ sessionFile: this.paths.sessionFile, extraArgs: this.engine.extraArgs });
this.pinRoot = pinRoot;
this.#checkSeal();
this.launcher = launcher;
this.verifier = verifier;
@@ -257,20 +325,168 @@ export class Controller {
if (this.barrier) await this.barrier(name, detail);
}
// The real file of the pi's bin, read now. No answer from the lookup, or a
// bin that no longer resolves, refuses: the lookup's own line names the
// step, never a path. The answer must still be the lookup's: the lookup run
// again on its bin gives the same root, dest, view and pkg. A stale
// answer (a link moved since the lookup) or one changed by hand refuses
// unsealed-engine.
#piFile() {
const pi = this.pi;
if (!pi || typeof pi !== "object" || typeof pi.bin !== "string" || !pi.bin) {
// The lookup's problem lines are words and its package's name; a
// hand-built one with a path or a control character in it isn't shown.
const problem = pi?.problem;
const plain = typeof problem === "string" && problem && !/[\x00-\x1f\x7f-\x9f\u2028\u2029]/.test(problem) && !problem.replaceAll(PACKAGE, "").includes("/");
const why = plain ? problem : "the pi lookup gave no binary";
throw new ControlRefusal(ENGINE_FEATURE_MISSING, why);
}
const label = this.#label();
let real;
try {
real = realpathSync(pi.bin);
} catch {
throw new ControlRefusal(ENGINE_FEATURE_MISSING, `${label} no longer resolves to a file`);
}
// The lookup run again on the same bin must give the same answer: what the
// fit check mounts (root at dest), runs (view) and digests (pkg) is then
// the package the launch runs.
const again = locate({ MOSAIC_PI_BIN: pi.bin, ...(typeof process.env.HOME === "string" ? { HOME: process.env.HOME } : {}) });
if (again.bin === null || ANSWER_KEYS.some((k) => (again[k] ?? null) !== (pi[k] ?? null))) {
throw new ControlRefusal(UNSEALED_ENGINE, `${label} no longer matches the lookup's answer; look the pi up again`);
}
return real;
}
// #1873 S4: the launch is this Node running the bin's real file, while the
// fit check runs the bin as a program, so its first line picks what runs it.
// Only a file whose first line runs node (#!/usr/bin/env node, or an
// absolute path to this same node) runs the same way in both.
#nodeScript(file) {
let head = "";
let fd = null;
try {
fd = openSync(file, "r");
const buf = Buffer.alloc(256);
head = buf.subarray(0, readSync(fd, buf, 0, buf.length, 0)).toString("latin1");
} catch {
head = "";
} finally {
if (fd !== null) try { closeSync(fd); } catch {}
}
const label = this.#label();
// The kernel reads at most 256 bytes for the first line; one that doesn't
// end inside them isn't run as a #! line at all.
const nl = head.indexOf("\n");
const m = nl < 0 ? null : NODE_SHEBANG.exec(head.slice(0, nl));
if (!m) throw new ControlRefusal(ENGINE_FEATURE_MISSING, `${label} isn't a node script, and the engine runs pi with node`);
// An absolute node must be this one: a file named node that is something
// else would let the fit check run another program than the launch does.
if (m[1] !== undefined) {
let interp = null;
try { interp = realpathSync(m[1]); } catch {}
if (interp === null || interp !== realpathSync(process.execPath)) {
throw new ControlRefusal(ENGINE_FEATURE_MISSING, `${label}'s first line runs another node than the engine's`);
}
return;
}
// env takes node from the fit check's PATH, the one a bus session gets
// (sessionEnv: the repository's .bin, then this process's PATH), and it
// looks inside the sandbox, which shows only some of the host. Up to the
// first node, every entry must read the same in both: absolute, already
// in plain form (a ".." after a link goes where the link points, not
// where the text does), and nowhere the sandbox gives its own copy of
// (/proc, /dev), in its spelling or its real path. An entry or a
// node there that can't be checked refuses. No node the sandbox can't show
// may come first, and the first one must be this node. No node on the
// PATH may come from the package the fit check mounts.
const engineNode = realpathSync(process.execPath);
const realOrNull = (p) => { try { return realpathSync(p); } catch { return null; } };
const root = realOrNull(this.pi.root) ?? this.pi.root;
const refuse = (why) => { throw new ControlRefusal(ENGINE_FEATURE_MISSING, `${label}'s first line runs node from the fit check's PATH, ${why}`); };
const missing = (e) => e?.code === "ENOENT" || e?.code === "ENOTDIR";
const ownCopy = (p) => OWN_COPY_DIRS.some((d) => p === d || p.startsWith(d + "/"));
let first = null;
for (const dir of String(sessionEnv(process.env, "fit").PATH ?? "").split(":")) {
if (first === null) {
if (!dir.startsWith("/")) refuse("which has an empty or relative entry");
const plain = dir.length > 1 ? dir.replace(/\/+$/, "") : dir;
if (resolve(plain) !== plain) refuse("which has an entry that isn't in plain form");
const own = "which has an entry the fit check's sandbox has its own copy of";
if (ownCopy(plain)) refuse(own);
// An entry that doesn't exist is judged by the real path of the
// nearest part of it that does.
let realDir = null;
for (let at = plain; realDir === null; at = dirname(at)) {
try { realDir = realpathSync(at); } catch (e) { if (!missing(e) || at === "/") refuse("where an entry can't be checked"); }
}
if (ownCopy(realDir)) refuse(own);
const candidate = join(plain, "node");
try { lstatSync(candidate); } catch (e) { if (missing(e)) continue; refuse("where an entry can't be checked"); }
let held = null;
try { held = statSync(candidate); } catch { refuse("where a node can't be checked"); }
if (!held.isFile() || !executable(candidate)) continue;
} else if (!dir.startsWith("/") || !executable(resolve(dir, "node"))) {
continue;
}
const candidate = resolve(dir, "node");
const real = realOrNull(candidate);
if (real === null) {
if (first === null) refuse("where a node can't be checked");
continue;
}
if (real === root || real.startsWith(root + "/")) {
throw new ControlRefusal(ENGINE_FEATURE_MISSING, `${label}'s package puts a node on the fit check's PATH`);
}
if (first !== null) continue;
first = shownInFit(candidate);
if (first === null) refuse("where a node the fit check can't see comes first");
}
if (first === null || realOrNull(first) !== engineNode) {
throw new ControlRefusal(ENGINE_FEATURE_MISSING, `${label}'s first line runs the first node on the fit check's PATH, which isn't the engine's`);
}
}
// The name a refusal line uses for the pi: the lookup's label, which names
// a step in words ("MOSAIC_PI_BIN", "the pi on PATH", "the repository's
// pi"), never a path. Anything else, such as a hand-built answer's label
// with a / or a newline in it, is "the pi".
#label() {
const label = this.pi?.label;
return typeof label === "string" && /^[A-Za-z_' ]{1,64}$/.test(label) ? label : "the pi";
}
// The seal covers the command: unless a test engine was given, the launch
// is this Node running the pinned Pi's bin, and nothing else.
// is this Node running the real file of the bin the lookup found, read
// again at each check, and nothing else.
#checkSeal() {
if (this.engine.sealed && (this.engine.command !== process.execPath || this.engine.preArgs.length !== 1 || this.engine.preArgs[0] !== join(this.pinRoot, PI_BIN))) {
throw new ControlRefusal(UNSEALED_ENGINE, "the engine command is not the pinned Pi");
if (this.engine.sealed && (this.engine.command !== process.execPath || this.engine.preArgs.length !== 1 || this.engine.preArgs[0] !== this.#piFile())) {
throw new ControlRefusal(UNSEALED_ENGINE, `the engine command is not ${this.#label()} the lookup found`);
}
const bad = this.engine.preArgs.find((a) => typeof a !== "string" || a === "-e" || a === "--extension" || a.startsWith("--extension="));
if (bad !== undefined) throw new ControlRefusal(UNSEALED_ENGINE, `engine pre-arguments carry ${bad}`);
checkSeal(this.piArgs);
checkSeal(this.piArgs, { noMcp: this.noMcp });
}
#pins() {
const pin = checkEnginePin(this.pinRoot);
return { engineVersion: pin.version, enginePin: pin.pin, argvDigest: argvDigest(this.engine.command, [...this.engine.preArgs, ...this.piArgs]) };
// #1873 S4: the fit check (pi-fit.mjs, cached by the digest of the files
// that run) stands where the version pin stood. A pi that fails it, or
// can't be checked, refuses engine-feature-missing with the check's line.
// engineVersion is the --version text (null when pi printed none) and
// enginePin the digest. --no-mcp joins the seal when the help lists it.
async #pins() {
this.#nodeScript(this.#piFile());
const fit = await piFit({ ...this.pi, label: this.#label() }, { levels: levels() });
if (!fit?.ok) throw new ControlRefusal(ENGINE_FEATURE_MISSING, typeof fit?.message === "string" ? fit.message : "the pi could not be checked");
if (typeof fit.digest !== "string" || !/^[0-9a-f]{64}$/.test(fit.digest)) throw new ControlRefusal(ENGINE_FEATURE_MISSING, "the fit check gave no digest");
this.noMcp = fit.noMcp === true;
this.piArgs = buildPiArgs({ sessionFile: this.paths.sessionFile, extraArgs: this.engine.extraArgs, noMcp: this.noMcp });
this.#checkSeal();
return { engineVersion: typeof fit.version === "string" ? fit.version : null, enginePin: fit.digest, argvDigest: argvDigest(this.engine.command, [...this.engine.preArgs, ...this.piArgs]) };
}
// The line for a pins mismatch against a claim's recorded pins.
static pinLine(recorded, since) {
return oldPinRecord(recorded) ? OLD_PIN_LINE : `the engine pin or launch argv changed since ${since}`;
}
// Every read after construction refuses a header ID other than the one the
@@ -341,7 +557,7 @@ export class Controller {
async start() {
this.#guardCheck("bind");
this.#checkSeal();
const pins = this.#pins();
const pins = await this.#pins();
const session = this.#readSession();
const classified = await this.store.classify(this.seatK, this.sessionK, {
units: this.units,
@@ -373,7 +589,7 @@ export class Controller {
let priorRef = null;
if (prior.n > 0 && !prior.damaged && prior.record.state === "stopped") {
const p = prior.record;
if (!equal(p.pins, pins)) throw new ControlRefusal(ENGINE_PIN_MISMATCH, "the engine pin or launch argv changed since the proven stop");
if (!equal(p.pins, pins)) throw new ControlRefusal(ENGINE_PIN_MISMATCH, Controller.pinLine(p.pins, "the proven stop"));
const leaf = p.leafAtProof !== undefined ? p.leafAtProof : p.leaf;
const branch = p.branchAtProof ?? p.branch;
if (leaf !== session.leaf || branch !== session.branch) throw new ControlRefusal("target", "the session branch or leaf changed since the proven stop");
@@ -1679,11 +1895,11 @@ export class Controller {
if (b.state !== "stopped" || b.admission !== "closed" || !this.#stopped(s)) return refused("stop-proof");
let pins;
try {
pins = this.#pins();
pins = await this.#pins();
} catch (err) {
return refused(err.code ?? ENGINE_PIN_MISMATCH);
return refused(err.code ?? ENGINE_PIN_MISMATCH, typeof err?.message === "string" ? { detail: err.message } : {});
}
if (!equal(pins, this.claim.record.pins)) return refused(ENGINE_PIN_MISMATCH);
if (!equal(pins, this.claim.record.pins)) return refused(ENGINE_PIN_MISMATCH, { detail: Controller.pinLine(this.claim.record.pins, "the proven stop") });
const session = this.#readSession();
if (session.leaf !== this.claim.record.leafAtProof || session.branch !== this.claim.record.branchAtProof) return refused("target");
if (!this.#checkConfirmation(r, c, "recover")) return refused("confirmation");
@@ -1715,7 +1931,7 @@ export class Controller {
throw new ControlRefusal(ELIGIBILITY, "the reserved claim changed since eligibility");
}
}
const pins = this.#pins();
const pins = await this.#pins();
if (!equal(pins, e.pins)) throw new ControlRefusal(ENGINE_PIN_MISMATCH, "the engine pin or launch argv changed since eligibility");
const session = this.#readSession();
if (session.leaf !== e.leaf || session.branch !== e.branch) throw new ControlRefusal("target", "the session branch or leaf changed since eligibility (K18)");
@@ -1,14 +1,21 @@
// The Pi pin and the engine seal (#1507, CHAT-03 §3, lead decisions 31–32).
// The engine seal and the engine's pins (#1507, CHAT-03 §3, lead decisions
// 31–32; #1873 pi-unpin S4).
//
// Pin: package-lock.json and npm's installed record
// (node_modules/.package-lock.json) must both name the pinned version with the
// pinned integrity. That ties the install to the package through npm's record;
// it is not a hash of the files on disk. `pi` runs dist/bundle/cli.js, the
// package's bin, and the built-in llama.cpp extension ships inside it.
// Pins: no pi version is pinned (Jason, 2026-11-19). The controller runs the
// pi binary.mjs's lookup finds (MOSAIC_PI_BIN, else the first pi on PATH, else
// the repository's) once the fit check (packages/harness/src/pi-fit.mjs)
// passes it. The claim's pins are what that check reports: `engineVersion`,
// the `--version` text (null when pi printed none), and `enginePin`, the
// sha256 digest of the files that run. A pi that fails the check, or can't be
// checked, is refused `engine-feature-missing`; a digest or argv that differs
// from a claim's is refused `engine-pin-mismatch`. A claim made before #1873
// holds npm's sha512 integrity in `enginePin`, which no digest equals, so it
// refuses once with OLD_PIN_LINE.
//
// Seal: the controller builds the launch argv. It always carries
// --no-extensions, --no-prompt-templates, --no-themes and --no-approve, and
// never an --extension argument (cli/args.js; usage.md 224 and 233–236).
// --no-extensions, --no-prompt-templates, --no-themes and --no-approve, plus
// --no-mcp when the pi's --help lists it, and never an --extension argument
// (cli/args.js; usage.md 224 and 233–236).
// --no-approve sets the project trust override to false, so a project's
// .pi/settings.json, SYSTEM.md, APPEND_SYSTEM.md and skills don't load even
// when trust.json under the agent dir trusts it (main.js 574–581;
@@ -23,60 +30,45 @@
// file, so the argv must be exactly the controller's prefix followed by
// ENGINE_OPTIONS pairs, each at most once with one plain value.
import { readFileSync } from "node:fs";
import { isAbsolute, join } from "node:path";
import { isAbsolute } from "node:path";
import { createHash } from "node:crypto";
import { ControlRefusal } from "./safe-fs.mjs";
export const PI_PACKAGE = "@earendil-works/pi-coding-agent";
export const PI_VERSION = "0.85.1";
export const PI_INTEGRITY = "sha512-FGRN+OHbWaefBPGaTggAdLjrIHW+s2PzLyglz/5dfLzb9of7uuXMXYC0fJIeZTw+shS32o2cuQ9jF7YSDuL/oQ==";
export const PI_BIN = join("node_modules", PI_PACKAGE, "dist", "bundle", "cli.js");
export const SEAL_FLAGS = Object.freeze(["--no-extensions", "--no-prompt-templates", "--no-themes", "--no-approve"]);
export const ENGINE_OPTIONS = Object.freeze(["--model", "--provider", "--thinking"]);
export const ENGINE_PIN_MISMATCH = "engine-pin-mismatch";
export const UNSEALED_ENGINE = "unsealed-engine";
export const ENGINE_FEATURE_MISSING = "engine-feature-missing";
function lockEntry(path) {
let lock;
try {
lock = JSON.parse(readFileSync(path, "utf8"));
} catch {
return null;
}
const entry = lock?.packages?.[`node_modules/${PI_PACKAGE}`];
return entry && typeof entry === "object" ? entry : null;
// The seal flags for a pi whose --help does (noMcp) or doesn't list --no-mcp.
export const sealFlags = (noMcp = false) => (noMcp ? [...SEAL_FLAGS, "--no-mcp"] : [...SEAL_FLAGS]);
// An `enginePin` the fit check gives is a sha256 hex digest.
export const PIN_DIGEST = /^[0-9a-f]{64}$/;
export const OLD_PIN_LINE = "the claim was made under the old pin record; start a new claim";
// True when a claim's pins came from the old npm pin record: an `enginePin`
// string that isn't a digest (npm's integrity is `sha512-...`).
export const oldPinRecord = (pins) => typeof pins?.enginePin === "string" && !PIN_DIGEST.test(pins.enginePin);
export function buildPiArgs({ sessionFile, extraArgs = [], noMcp = false }) {
return ["--mode", "rpc", ...sealFlags(noMcp), "--session", sessionFile, ...extraArgs];
}
// `root` holds package-lock.json and node_modules/.package-lock.json.
export function checkEnginePin(root) {
for (const path of [join(root, "package-lock.json"), join(root, "node_modules", ".package-lock.json")]) {
const entry = lockEntry(path);
if (!entry || entry.version !== PI_VERSION || entry.integrity !== PI_INTEGRITY) {
throw new ControlRefusal(ENGINE_PIN_MISMATCH, `${path} does not pin ${PI_PACKAGE} ${PI_VERSION} with the pinned integrity`);
}
}
return { version: PI_VERSION, pin: PI_INTEGRITY };
}
export function buildPiArgs({ sessionFile, extraArgs = [] }) {
return ["--mode", "rpc", ...SEAL_FLAGS, "--session", sessionFile, ...extraArgs];
}
// Refuses any argv that is not `--mode rpc`, the four --no-* flags and
// `--session <absolute path>`, in that order, followed by ENGINE_OPTIONS
// pairs. That covers --extension in either spelling, a second --mode or
// Refuses any argv that is not `--mode rpc`, the four --no-* flags (and
// --no-mcp when noMcp), and `--session <absolute path>`, in that order,
// followed by ENGINE_OPTIONS pairs. That covers --extension in either spelling, a second --mode or
// --session, session and output flags (--no-session, --fork, --export, ...)
// and stray prompt words.
export function checkSeal(args) {
export function checkSeal(args, { noMcp = false } = {}) {
if (!Array.isArray(args) || args.some((a) => typeof a !== "string")) throw new ControlRefusal(UNSEALED_ENGINE, "launch argv is not a list of strings");
const extension = args.find((a) => a === "-e" || a === "--extension" || a.startsWith("--extension="));
if (extension !== undefined) throw new ControlRefusal(UNSEALED_ENGINE, `launch argv carries ${extension}`);
for (const flag of SEAL_FLAGS) {
const flags = sealFlags(noMcp);
for (const flag of flags) {
if (!args.includes(flag)) throw new ControlRefusal(UNSEALED_ENGINE, `launch argv lacks ${flag}`);
}
const prefix = ["--mode", "rpc", ...SEAL_FLAGS, "--session"];
const prefix = ["--mode", "rpc", ...flags, "--session"];
if (prefix.some((a, i) => args[i] !== a)) throw new ControlRefusal(UNSEALED_ENGINE, `launch argv does not start with ${prefix.join(" ")}`);
const file = args[prefix.length];
if (typeof file !== "string" || !isAbsolute(file)) throw new ControlRefusal(UNSEALED_ENGINE, "the --session value is not an absolute path");
+1 -1
View File
@@ -1,6 +1,6 @@
// Run tracking, overlap signals and receipt settlement (#1507, CHAT-03 §3).
//
// Pinned Pi ties no event to a prompt. Under the seal (pi-pin.mjs) the Mosaic
// Pi ties no event to a prompt. Under the seal (pi-seal.mjs) the Mosaic
// prompt in the slot is the only thing that can start a run, so a run that
// follows the slot's ack is attributed to it by order. The tracker watches
// for signs that the seal failed (O1–O6) and for lead decision 34's
+414 -12
View File
@@ -5,23 +5,27 @@
import { test, after } from "node:test";
import assert from "node:assert/strict";
import { appendFileSync, copyFileSync, linkSync, mkdirSync, statSync, mkdtempSync, readFileSync, readdirSync, rmSync, symlinkSync, unlinkSync, writeFileSync, lstatSync } from "node:fs";
import { appendFileSync, chmodSync, copyFileSync, linkSync, mkdirSync, statSync, mkdtempSync, readFileSync, readdirSync, realpathSync, rmSync, symlinkSync, unlinkSync, writeFileSync, lstatSync } from "node:fs";
import { homedir, tmpdir } from "node:os";
import { join } from "node:path";
import { dirname, join } from "node:path";
import { spawnSync } from "node:child_process";
import { createHash } from "node:crypto";
import { types as utilTypes } from "node:util";
import { ALREADY_ACTIVE, ClaimStore, FOREIGN_HOST, UNSAFE_REPLACEMENT, held, machineId } from "../src/claim.mjs";
import { Controller } from "../src/controller.mjs";
import { Controller, REPO_ROOT } from "../src/controller.mjs";
import { ConversationClient } from "../src/client.mjs";
import { AUTHORITY, scopeAvailable } from "../src/cohort.mjs";
import { FixtureVerifier } from "../src/records.mjs";
import { LIVE_SESSION_REFUSED } from "../src/guard.mjs";
import { ENGINE_PIN_MISMATCH, PI_PACKAGE } from "../src/pi-pin.mjs";
import { ENGINE_FEATURE_MISSING, ENGINE_PIN_MISMATCH, OLD_PIN_LINE, UNSEALED_ENGINE } from "../src/pi-seal.mjs";
import { piFit } from "../../harness/src/pi-fit.mjs";
import { levels } from "../../plugins/harness/pi/index.mjs";
import { bootId } from "../../discord/src/journal.mjs";
import { FakeLauncher } from "./fake-pi.mjs";
import {
REPO, FAST, fixture, controllerFor, started, spawnController, killChildren, cleanupAll, reap, claimRecords,
userEntry, assistantEntry, thinkingEntry, receiptState, noUnits, tick, killShims, shimsGone,
PI_PACKAGE, standInPi, piAt,
} from "./harness.mjs";
const reaped = [];
@@ -579,14 +583,8 @@ test("W8: resume after a proven stop with the same pins: new claim ID, generatio
test("W9: resume with a changed binary, argv digest, branch or leaf is refused and the claim is unchanged", async () => {
const cases = {
binary: (fx) => {
const root = mkdtempSync(join(fx.base, "pin-"));
const lock = { packages: { [`node_modules/${PI_PACKAGE}`]: { version: "0.85.2", integrity: "sha512-other" } } };
mkdirSync(join(root, "node_modules"));
writeFileSync(join(root, "package-lock.json"), JSON.stringify(lock));
writeFileSync(join(root, "node_modules", ".package-lock.json"), JSON.stringify(lock));
return [{ pinRoot: root }, ENGINE_PIN_MISMATCH];
},
// Another pi, which the fit check passes, with other files.
binary: (fx) => [{ pi: standInPi(join(fx.base, "other-pi"), { version: "0.85.2" }).found }, ENGINE_PIN_MISMATCH],
argv: () => [{ engine: { extraArgs: ["--model", "other"] } }, ENGINE_PIN_MISMATCH],
leaf: (fx) => {
appendFileSync(fx.sessionFile, JSON.stringify(userEntry("d4e5f6a7", "b2c3d4e5", "outside", 9)) + "\n");
@@ -610,6 +608,410 @@ test("W9: resume with a changed binary, argv digest, branch or leaf is refused a
}
});
// ---- #1873 pi-unpin S4: the engine is the pi the lookup finds --------------
test("S4a: a stand-in pi with another version and every feature launches, and the claim records its version and digest", async () => {
const fx = fixture();
const pi = standInPi(join(fx.base, "pi"), { version: "7.4.2" });
const h = await started({ fx, pi: pi.found });
try {
assert.equal(h.launcher.launches.length, 1);
const fit = await piFit(pi.found, { levels: levels() });
assert.equal(fit.ok, true, fit.message);
const recorded = h.ctrl.claim.record.pins;
assert.equal(recorded.engineVersion, "7.4.2");
assert.equal(recorded.enginePin, fit.digest);
assert.match(recorded.enginePin, /^[0-9a-f]{64}$/);
} finally {
await h.close();
}
});
test("S4b: a stand-in pi whose --help lacks --mode rpc is refused engine-feature-missing with a plain line; nothing launches or is claimed", async () => {
const fx = fixture();
const pi = standInPi(join(fx.base, "pi"), { modes: ["text", "json"] });
const k = keysFor(fx, { pi: pi.found });
const { ctrl, launcher } = controllerFor(fx, { pi: pi.found });
try {
await assert.rejects(ctrl.start(), (e) => {
assert.equal(e.code, ENGINE_FEATURE_MISSING);
assert.equal(e.message, "pi 9.9.9 (MOSAIC_PI_BIN) lacks --mode rpc");
return true;
});
assert.equal(launcher.launches.length, 0);
assert.deepEqual([revisionTexts(k.store, k.seatK), revisionTexts(k.store, k.sessionK)], [[], []]);
} finally {
await ctrl.close().catch(() => {});
}
});
test("S4c: changed bytes in the pi between launch and resume refuse engine-pin-mismatch; the claim is unchanged", async () => {
const fx = fixture();
const pi = standInPi(join(fx.base, "pi"));
const first = await provenStop(fx, { pi: pi.found });
appendFileSync(pi.cli, "// one more line\n");
const k = keysFor(fx, { pi: pi.found });
const before = [revisionTexts(k.store, k.seatK), revisionTexts(k.store, k.sessionK)];
const { ctrl, launcher } = controllerFor(fx, { pi: piAt(pi.bin) });
try {
await assert.rejects(ctrl.start(), (e) => {
assert.equal(e.code, ENGINE_PIN_MISMATCH);
assert.equal(e.message, "the engine pin or launch argv changed since the proven stop");
return true;
});
assert.equal(launcher.launches.length, 0);
assert.deepEqual([revisionTexts(k.store, k.seatK), revisionTexts(k.store, k.sessionK)], before);
assert.equal(first.pins.engineVersion, "9.9.9");
} finally {
await ctrl.close().catch(() => {});
}
});
test("S4d: a pre-argument that is not the resolved bin's real file refuses unsealed-engine at bind, as does a bin link moved after construction", async () => {
const fx = fixture();
const pi = standInPi(join(fx.base, "pi"));
const make = () => new Controller({ fixtureRoot: fx.base, claimRoot: fx.claimRoot, socketDir: fx.socketDir, sessionFile: fx.sessionFile, seat: fx.seat, launcher: new FakeLauncher(), units: noUnits, timeouts: FAST, pi: pi.found });
{
const ctrl = make();
try {
assert.deepEqual(ctrl.engine.preArgs, [realpathSync(pi.cli)]);
// The repository's own pi, which the old pin accepted.
ctrl.engine.preArgs = [join(REPO_ROOT, "node_modules", PI_PACKAGE, "dist", "bundle", "cli.js")];
await assert.rejects(ctrl.start(), (e) => e.code === UNSEALED_ENGINE);
assert.equal(ctrl.launcher.launches.length, 0);
// The bin link itself, not its real file.
ctrl.engine.preArgs = [pi.bin];
await assert.rejects(ctrl.start(), (e) => e.code === UNSEALED_ENGINE);
assert.equal(ctrl.launcher.launches.length, 0);
} finally {
await ctrl.close().catch(() => {});
}
}
{
const ctrl = make();
const other = standInPi(join(fx.base, "pi-other"), { version: "1.0.0" });
try {
unlinkSync(pi.bin);
symlinkSync(other.cli, pi.bin);
await assert.rejects(ctrl.start(), (e) => e.code === UNSEALED_ENGINE);
assert.equal(ctrl.launcher.launches.length, 0);
} finally {
await ctrl.close().catch(() => {});
}
}
});
test("S4e: a claim made under the old pin record (npm's integrity in enginePin) refuses engine-pin-mismatch with the old-record line at every start of that session", async () => {
const fx = fixture();
const first = await provenStop(fx);
// Rewrite the stopped heads as a claim made before #1873 would hold them.
for (const { path, record } of claimRecords(fx)) {
if (record?.claimId !== first.claimId || record.state !== "stopped") continue;
const old = { ...record, pins: { ...record.pins, engineVersion: "0.85.1", enginePin: "sha512-FGRN+OHbWaefBPGaTggAdLjrIHW+s2PzLyglz/5dfLzb9of7uuXMXYC0fJIeZTw+shS32o2cuQ9jF7YSDuL/oQ==" } };
unlinkSync(path);
writeFileSync(path, JSON.stringify(old), { mode: 0o400 });
}
// Like any pin mismatch, the refusal holds at each start: the session's last
// proven stop is what a resume is checked against.
for (let i = 0; i < 2; i++) {
const { ctrl, launcher } = controllerFor(fx);
try {
await assert.rejects(ctrl.start(), (e) => {
assert.equal(e.code, ENGINE_PIN_MISMATCH);
assert.equal(e.message, OLD_PIN_LINE);
assert.match(e.message, /old pin record; start a new claim/);
return true;
});
assert.equal(launcher.launches.length, 0);
} finally {
await ctrl.close().catch(() => {});
}
}
});
test("S4f: a pi whose first line doesn't run node is refused engine-feature-missing, though the fit check passes it; #!/usr/bin/env node launches", async () => {
const fx = fixture();
// The sh form of the stand-in passes the fit check, which runs the bin as a
// program, but the engine runs the bin's real file with node.
const sh = standInPi(join(fx.base, "pi-sh"), { form: "sh" });
const fit = await piFit(sh.found, { levels: levels() });
assert.equal(fit.ok, true, fit.message);
const k = keysFor(fx, { pi: sh.found });
{
const { ctrl, launcher } = controllerFor(fx, { pi: sh.found });
try {
await assert.rejects(ctrl.start(), (e) => {
assert.equal(e.code, ENGINE_FEATURE_MISSING);
assert.equal(e.message, "MOSAIC_PI_BIN isn't a node script, and the engine runs pi with node");
return true;
});
assert.equal(launcher.launches.length, 0);
assert.deepEqual([revisionTexts(k.store, k.seatK), revisionTexts(k.store, k.sessionK)], [[], []]);
} finally {
await ctrl.close().catch(() => {});
}
}
// The node form with the shebang npm writes for pi.
const env = standInPi(join(fx.base, "pi-env"));
const text = readFileSync(env.cli, "utf8");
writeFileSync(env.cli, "#!/usr/bin/env node" + text.slice(text.indexOf("\n")));
const h = await started({ fx, pi: env.found });
try {
assert.equal(h.launcher.launches.length, 1);
assert.equal(h.launcher.launches[0].args[0], realpathSync(env.cli));
} finally {
await h.close();
}
});
test("S4g: a lookup answer that is no longer the lookup's refuses unsealed-engine: a package link moved since the lookup, a hand-built answer with another bin, one field changed by hand, a bin no package holds, or a getter; an answer that changes after construction changes nothing", async () => {
const fx = fixture();
const good = standInPi(join(fx.base, "pi-good"));
const lacking = standInPi(join(fx.base, "pi-lacking"), { modes: ["text", "json"] });
// A global prefix whose package directory is a link, as a version manager
// keeps it.
const prefix = join(fx.base, "prefix");
mkdirSync(join(prefix, "lib", "node_modules", "@earendil-works"), { recursive: true });
mkdirSync(join(prefix, "bin"));
const link = join(prefix, "lib", "node_modules", PI_PACKAGE);
symlinkSync(good.pkg, link);
const bin = join(prefix, "bin", "pi");
symlinkSync(`../lib/node_modules/${PI_PACKAGE}/dist/bundle/cli.js`, bin);
const found = piAt(bin);
assert.equal(found.bin, bin);
const make = (pi) => new Controller({ fixtureRoot: fx.base, claimRoot: fx.claimRoot, socketDir: fx.socketDir, sessionFile: fx.sessionFile, seat: fx.seat, launcher: new FakeLauncher(), units: noUnits, timeouts: FAST, pi });
const ctrl = make(found);
try {
unlinkSync(link);
symlinkSync(lacking.pkg, link);
// Built before the move: the start refuses.
await assert.rejects(ctrl.start(), (e) => {
assert.equal(e.code, UNSEALED_ENGINE);
assert.equal(e.message, "MOSAIC_PI_BIN no longer matches the lookup's answer; look the pi up again");
return true;
});
assert.equal(ctrl.launcher.launches.length, 0);
} finally {
await ctrl.close().catch(() => {});
}
// Built after the move, with the stale answer.
assert.throws(() => make(found), (e) => e.code === UNSEALED_ENGINE);
// An answer with a bin swapped in by hand.
assert.throws(() => make({ ...good.found, bin: lacking.bin }), (e) => e.code === UNSEALED_ENGINE);
assert.throws(() => make({ bin: good.bin, label: "MOSAIC_PI_BIN" }), (e) => e.code === UNSEALED_ENGINE);
// One field changed by hand at a time: the fit check would mount, run or
// digest another package than the launch runs.
for (const k of ["root", "dest", "view", "pkg"]) {
const value = k === "dest" ? lacking.found.root : lacking.found[k];
assert.throws(() => make({ ...good.found, [k]: value }), (e) => e.code === UNSEALED_ENGINE, k);
}
// The lacking pi's answer with the good package mounted over its path.
assert.throws(() => make({ ...lacking.found, root: good.found.root, dest: lacking.found.root }), (e) => e.code === UNSEALED_ENGINE);
// A bin the lookup no longer takes (no package holds it) refuses when the
// controller is built.
const loose = join(fx.base, "loose-pi");
writeFileSync(loose, "#!/usr/bin/env node\n", { mode: 0o755 });
assert.throws(() => make({ bin: loose, label: "MOSAIC_PI_BIN" }), (e) => e.code === UNSEALED_ENGINE);
// A getter is not a data field: the copy the controller keeps lacks it.
assert.throws(() => make({ ...good.found, get view() { return good.found.view; } }), (e) => e.code === UNSEALED_ENGINE);
// An answer that changes after the controller is built changes nothing: it
// keeps a frozen plain copy, and the start checks and launches the good pi.
let flipped = false;
const live = new Proxy({ ...good.found }, {
get: (t, k) => (flipped && k in lacking.found ? lacking.found[k] : t[k]),
getOwnPropertyDescriptor: (t, k) => {
const d = Reflect.getOwnPropertyDescriptor(t, k);
return d && flipped && k in lacking.found ? { ...d, value: lacking.found[k] } : d;
},
});
const { ctrl: kept, launcher: keptLauncher } = controllerFor(fx, { pi: live });
try {
flipped = true;
assert.ok(Object.isFrozen(kept.pi));
assert.equal(utilTypes.isProxy(kept.pi), false);
assert.deepEqual(kept.pi, good.found);
await kept.start();
assert.equal(keptLauncher.launches.length, 1);
assert.equal(keptLauncher.launches[0].args[0], realpathSync(good.cli));
} finally {
await kept.close({ killEngine: true }).catch(() => {});
}
// The answer as the lookup gives it still builds.
await make(piAt(good.bin)).close();
});
test("S4h: a first line the kernel and node read differently refuses engine-feature-missing: an absolute node that isn't the engine's, a first line past 256 bytes, a CR before the newline, under env a node from the package, a node the fit check's sandbox can't see ahead of the first it can (even a link to the engine's node), an empty, relative or unplain entry before the first node, one in /proc or /dev, one that can't be checked, or a first shown node that isn't the engine's; the engine's own directory first launches, and a label that isn't the lookup's kind reads \"the pi\", in the fit check's line too; a lookup's problem line shows, a hand-built one with a path or a newline doesn't", async () => {
const fx = fixture();
const refused = async (pi, message, f = fx) => {
const { ctrl, launcher } = controllerFor(f, { pi });
try {
await assert.rejects(ctrl.start(), (e) => {
assert.equal(e.code, ENGINE_FEATURE_MISSING);
assert.equal(e.message, message);
return true;
});
assert.equal(launcher.launches.length, 0);
} finally {
await ctrl.close().catch(() => {});
}
};
const withFirstLine = (name, line) => {
const pi = standInPi(join(fx.base, name));
const text = readFileSync(pi.cli, "utf8");
writeFileSync(pi.cli, line + text.slice(text.indexOf("\n")));
return pi;
};
// A program named node in the package that runs another program: here the
// real node, so the fit check passes it.
const shim = standInPi(join(fx.base, "pi-shim"));
const fakeNode = join(dirname(shim.cli), "node");
writeFileSync(fakeNode, `#!/bin/sh\nexec ${process.execPath} "$@"\n`, { mode: 0o755 });
const shimText = readFileSync(shim.cli, "utf8");
writeFileSync(shim.cli, `#!${fakeNode}` + shimText.slice(shimText.indexOf("\n")));
const fit = await piFit(shim.found, { levels: levels() });
assert.equal(fit.ok, true, fit.message);
await refused(shim.found, "MOSAIC_PI_BIN's first line runs another node than the engine's");
// The kernel reads 256 bytes for the first line; this one ends past them.
const long = withFirstLine("pi-long", "#!/usr/bin/env node" + " ".repeat(260));
await refused(long.found, "MOSAIC_PI_BIN isn't a node script, and the engine runs pi with node");
// The kernel looks for a program named with the CR.
const cr = withFirstLine("pi-cr", "#!/usr/bin/env node\r");
await refused(cr.found, "MOSAIC_PI_BIN isn't a node script, and the engine runs pi with node");
// A link to the engine's own node is the same file.
const linked = standInPi(join(fx.base, "pi-linked"));
const nodeLink = join(dirname(linked.cli), "node");
symlinkSync(process.execPath, nodeLink);
const linkedText = readFileSync(linked.cli, "utf8");
writeFileSync(linked.cli, `#!${nodeLink}` + linkedText.slice(linkedText.indexOf("\n")));
const h = await started({ fx, pi: linked.found });
try {
assert.equal(h.launcher.launches.length, 1);
assert.equal(h.launcher.launches[0].args[0], realpathSync(linked.cli));
} finally {
await h.close();
}
// #!/usr/bin/env node takes node from the fit check's PATH, which is this
// process's PATH behind the repository's .bin.
const envPi = withFirstLine("pi-env-path", "#!/usr/bin/env node");
const fx2 = fixture();
const onPath = async (dir, fn) => {
const saved = process.env.PATH;
process.env.PATH = `${dir}:${saved}`;
try {
await fn();
} finally {
process.env.PATH = saved;
}
};
// A node in the package itself, put on PATH.
const pkgBin = join(envPi.pkg, "bin");
mkdirSync(pkgBin);
writeFileSync(join(pkgBin, "node"), `#!/bin/sh\nexec ${process.execPath} "$@"\n`, { mode: 0o755 });
await onPath(pkgBin, () => refused(envPi.found, "MOSAIC_PI_BIN's package puts a node on the fit check's PATH", fx2));
// The fit check's sandbox shows only /usr, /etc, the host's /bin, /lib,
// /lib64 and /sbin, and the engine's own node file. A node in any other
// directory ahead of the engine's: env in the sandbox would pass it by,
// while the host finds it first. Even a link to the engine's node.
const otherBin = join(fx.base, "other-bin");
mkdirSync(otherBin);
writeFileSync(join(otherBin, "node"), `#!/bin/sh\nexec ${process.execPath} "$@"\n`, { mode: 0o755 });
const hidden = "MOSAIC_PI_BIN's first line runs node from the fit check's PATH, where a node the fit check can't see comes first";
await onPath(otherBin, () => refused(envPi.found, hidden, fx2));
const linkBin = join(fx.base, "link-bin");
mkdirSync(linkBin);
symlinkSync(process.execPath, join(linkBin, "node"));
await onPath(linkBin, () => refused(envPi.found, hidden, fx2));
// env reads an empty or relative entry against the fit check's directory.
const relative = "MOSAIC_PI_BIN's first line runs node from the fit check's PATH, which has an empty or relative entry";
await onPath("", () => refused(envPi.found, relative, fx2));
await onPath("some/bin", () => refused(envPi.found, relative, fx2));
// Up to the first node, an entry must read the same in the sandbox as on
// the host. A ".." after a link goes where the link points, not where the
// text does.
symlinkSync("/usr/lib", join(fx.base, "sys"));
await onPath(`${fx.base}/sys/../bin`, () => refused(envPi.found, "MOSAIC_PI_BIN's first line runs node from the fit check's PATH, which has an entry that isn't in plain form", fx2));
// The sandbox makes its own /proc and /dev: pid 1 there is its own, and a
// link into /proc counts too, even past a pid the host lacks.
const ownCopy = "MOSAIC_PI_BIN's first line runs node from the fit check's PATH, which has an entry the fit check's sandbox has its own copy of";
await onPath("/proc/1/root/usr/bin", () => refused(envPi.found, ownCopy, fx2));
await onPath("/dev/mosaic-no-such-dir", () => refused(envPi.found, ownCopy, fx2));
symlinkSync("/proc", join(fx.base, "p"));
await onPath(`${fx.base}/p/999999999/root/usr/bin`, () => refused(envPi.found, ownCopy, fx2));
// A node link that leads nowhere on the host can't be checked: what the
// sandbox finds at its target need not be what the host lacks there.
const dangling = join(fx.base, "dangling-bin");
mkdirSync(dangling);
symlinkSync("/tmp/mosaic-no-such-dir/node", join(dangling, "node"));
await onPath(dangling, () => refused(envPi.found, "MOSAIC_PI_BIN's first line runs node from the fit check's PATH, where a node can't be checked", fx2));
// An entry the host can't read can't be checked.
const locked = join(fx.base, "locked");
mkdirSync(join(locked, "bin"), { recursive: true });
chmodSync(locked, 0o000);
try {
await onPath(join(locked, "bin"), () => refused(envPi.found, "MOSAIC_PI_BIN's first line runs node from the fit check's PATH, where an entry can't be checked", fx2));
} finally {
chmodSync(locked, 0o755);
}
// The first node the sandbox shows must be the engine's: no node at all
// isn't, and neither is a system node that is another file.
const notEngine = "MOSAIC_PI_BIN's first line runs the first node on the fit check's PATH, which isn't the engine's";
const withPath = async (path, fn) => {
const saved = process.env.PATH;
process.env.PATH = path;
try {
await fn();
} finally {
process.env.PATH = saved;
}
};
await withPath("/usr/share", () => refused(envPi.found, notEngine, fx2));
const systemNode = ["/usr/bin/node", "/usr/local/bin/node"].find((n) => {
try {
return realpathSync(n) !== realpathSync(process.execPath);
} catch {
return false;
}
});
if (systemNode) await withPath(`${dirname(systemNode)}:${dirname(process.execPath)}`, () => refused(envPi.found, notEngine, fx2));
// The engine's own directory first: the sandbox shows that file at the same
// path, so env runs the engine's node in both.
// Entries after it are never read, so a trailing slash on it and an empty
// entry at the end change nothing. The sandbox's /tmp starts empty, so a
// /tmp entry with no node on the host holds none there either.
await withPath(`/tmp/mosaic-no-such-dir:${dirname(process.execPath)}/:/usr/bin:/bin:`, async () => {
const h2 = await started({ fx: fx2, pi: envPi.found });
try {
assert.equal(h2.launcher.launches.length, 1);
assert.equal(h2.launcher.launches[0].args[0], realpathSync(envPi.cli));
} finally {
await h2.close();
}
});
// A hand-built label that is a path never reaches a refusal line.
await refused({ ...shim.found, label: join(fx.base, "named") }, "the pi's first line runs another node than the engine's");
await refused({ ...shim.found, label: "MOSAIC_PI_BIN\nrefused: forged" }, "the pi's first line runs another node than the engine's");
// Nor the fit check's own line.
const lacking = standInPi(join(fx.base, "pi-lacking"), { modes: ["text", "json"] });
await refused({ ...lacking.found, label: join(fx.base, "named") }, "pi 9.9.9 (the pi) lacks --mode rpc");
// A lookup that found no pi refuses when the controller is built: its own
// line, which names the step and the package, never a path; a hand-built
// line with a path or a newline isn't shown.
const unbuilt = (pi, message) =>
assert.throws(() => controllerFor(fx, { pi }), (e) => {
assert.equal(e.code, ENGINE_FEATURE_MISSING);
assert.equal(e.message, message);
return true;
});
unbuilt(piAt("relative/pi"), "MOSAIC_PI_BIN isn't an executable file at an absolute path");
const loose = join(fx.base, "loose-pi");
writeFileSync(loose, "#!/usr/bin/env node\n", { mode: 0o755 });
unbuilt(piAt(loose), `MOSAIC_PI_BIN isn't part of the ${PI_PACKAGE} package`);
unbuilt({ bin: null, problem: `${fx.base} has no pi` }, "the pi lookup gave no binary");
unbuilt({ bin: null, problem: "no pi\nrefused: forged" }, "the pi lookup gave no binary");
unbuilt({ bin: null, problem: "no pi\u2028refused: forged" }, "the pi lookup gave no binary");
unbuilt({ bin: null, problem: "no pi\u0085refused: forged" }, "the pi lookup gave no binary");
});
// ---- W11, W16, W17 ----------------------------------------------------------
test("W11: the controller writes no session file; only the fake engine's own appends appear", async () => {
+14 -22
View File
@@ -9,17 +9,17 @@
import { test, after } from "node:test";
import assert from "node:assert/strict";
import { appendFileSync, chmodSync, copyFileSync, mkdirSync, readFileSync, renameSync, rmdirSync, writeFileSync } from "node:fs";
import { appendFileSync, chmodSync, mkdirSync, readFileSync, renameSync, rmdirSync, writeFileSync } from "node:fs";
import { spawn, spawnSync } from "node:child_process";
import { dirname, join } from "node:path";
import { ClaimStore, FOREIGN_HOST, defaultHost, newClaimId, unitNameFor } from "../src/claim.mjs";
import { ConversationClient } from "../src/client.mjs";
import { AUTHORITY, PgroupLauncher, ScopeLauncher, engineExitCohort, forceStopCohort, releaseCohort, scopeAvailable, shimRequest, systemctlShow, systemdUnits } from "../src/cohort.mjs";
import { Controller, ELIGIBILITY, TEST_ENGINE } from "../src/controller.mjs";
import { ENGINE_ENV, ENGINE_PIN_MISMATCH, engineEnv } from "../src/pi-pin.mjs";
import { ENGINE_ENV, ENGINE_PIN_MISMATCH, engineEnv } from "../src/pi-seal.mjs";
import { FixtureVerifier, newId } from "../src/records.mjs";
import { ControlClient, FakeLauncher } from "./fake-pi.mjs";
import { FAST, REPO, assistantEntry, claimRecords, cleanupAll, controllerFor, fixture, killChildren, killShims, liveShims, noUnits, reap, receiptState, shimsGone, spawnController, started, tick } from "./harness.mjs";
import { FAST, PI, assistantEntry, claimRecords, cleanupAll, controllerFor, fixture, killChildren, killShims, liveShims, noUnits, reap, receiptState, shimsGone, spawnController, standInPi, started, tick } from "./harness.mjs";
const reaped = [];
const strays = new Set();
@@ -111,7 +111,7 @@ async function live({ kind = "scope", barrier = null, verifier = new FixtureVeri
fixtureRoot: fx.base, claimRoot: fx.claimRoot, socketDir: fx.socketDir, sessionFile: fx.sessionFile, seat: fx.seat,
launcher: launcher ?? (kind === "scope" ? new ScopeLauncher() : new PgroupLauncher()),
engine: { cwd: fx.proj }, [TEST_ENGINE]: { command: process.execPath, preArgs: [FAKE_PI], env: { ...process.env, FAKE_PI_CONTROL: control, FAKE_PI_LOG: join(fx.base, "fake.log") } },
verifier, units: kind === "scope" ? systemdUnits : noUnits, timeouts, barrier,
verifier, units: kind === "scope" ? systemdUnits : noUnits, timeouts, barrier, pi: PI.found,
});
await ctrl.start();
assert.equal(ctrl.binding.state, "active", JSON.stringify(ctrl.evidence.uncertain));
@@ -532,14 +532,6 @@ async function provenStop(h, c = h.client) {
return fs.stop.id;
}
function pinRootCopy(fx) {
const root = join(fx.base, "pins");
mkdirSync(join(root, "node_modules"), { recursive: true });
copyFileSync(join(REPO, "package-lock.json"), join(root, "package-lock.json"));
copyFileSync(join(REPO, "node_modules", ".package-lock.json"), join(root, "node_modules", ".package-lock.json"));
return root;
}
test("K6: recover without proof, without confirmation, or with changed pins is refused", async () => {
{
const h = await started();
@@ -562,8 +554,8 @@ test("K6: recover without proof, without confirmation, or with changed pins is r
}
{
const fx = track(fixture());
const pinRoot = pinRootCopy(fx);
const h = await started({ fx, pinRoot });
const pi = standInPi(join(fx.base, "pi"));
const h = await started({ fx, pi: pi.found });
try {
const stop = await provenStop(h);
const missing = await h.client.request("recover", { stop });
@@ -574,13 +566,13 @@ test("K6: recover without proof, without confirmation, or with changed pins is r
const id = issued.data.confirmation.id;
await h.client.request("answer-confirmation", { confirmation: id, answer: "confirm" });
assert.equal((await h.client.request("recover", { stop, confirmation: id })).refusal, "confirmation", "a confirmation for another operation");
const lock = join(pinRoot, "package-lock.json");
const original = readFileSync(lock, "utf8");
const changed = JSON.parse(original);
changed.packages["node_modules/@earendil-works/pi-coding-agent"].version = "0.0.0";
writeFileSync(lock, JSON.stringify(changed));
assert.equal((await h.client.confirmed("recover", { stop })).refusal, ENGINE_PIN_MISMATCH);
writeFileSync(lock, original);
// Other bytes in the pi the lookup finds, then the same bytes again.
const original = readFileSync(pi.cli, "utf8");
writeFileSync(pi.cli, `${original}// other bytes\n`);
const changed = await h.client.confirmed("recover", { stop });
assert.equal(changed.refusal, ENGINE_PIN_MISMATCH);
assert.equal(changed.detail, "the engine pin or launch argv changed since the proven stop");
writeFileSync(pi.cli, original);
const again = await h.client.request("issue-confirmation", { operationToConfirm: "recover" });
const once = again.data.confirmation.id;
await h.client.request("answer-confirmation", { confirmation: once, answer: "confirm" });
@@ -1177,7 +1169,7 @@ function scopeController(fx, { host, verifier = new FixtureVerifier({ authoritie
fixtureRoot: fx.base, claimRoot: fx.claimRoot, socketDir: fx.socketDir, sessionFile: fx.sessionFile, seat: fx.seat,
launcher, engine: { cwd: fx.proj },
[TEST_ENGINE]: { command: process.execPath, preArgs: [FAKE_PI], env: { ...process.env, FAKE_PI_CONTROL: join(fx.base, "fake-b.sock"), FAKE_PI_LOG: join(fx.base, "fake-b.log") } },
verifier, units: systemdUnits, timeouts: FAST, barrier, host,
verifier, units: systemdUnits, timeouts: FAST, barrier, host, pi: PI.found,
});
}
+2 -1
View File
@@ -20,6 +20,7 @@ import { Controller, TEST_ENGINE } from "../src/controller.mjs";
import { AUTHORITY, PgroupLauncher, ScopeLauncher, systemdUnits } from "../src/cohort.mjs";
import { FixtureVerifier } from "../src/records.mjs";
import { defaultHost } from "../src/claim.mjs";
import { locate } from "../../plugins/harness/pi/binary.mjs";
const cfg = JSON.parse(process.argv[2]);
const out = (v) => process.stdout.write(JSON.stringify(v) + "\n");
@@ -79,7 +80,7 @@ try {
engine: { cwd: fx.proj }, [TEST_ENGINE]: { command: process.execPath, preArgs: [fakePi], env: { ...process.env, ...(cfg.fakeEnv ?? {}) } },
verifier: cfg.verifier === false ? null : new FixtureVerifier({ authorities: [AUTHORITY] }),
units: cfg.units === "absent" ? { lookup: async () => ({ state: "absent" }) } : systemdUnits,
host, barrier, timeouts: cfg.timeouts,
host, barrier, timeouts: cfg.timeouts, pi: locate({ MOSAIC_PI_BIN: cfg.piBin, PATH: "/usr/bin:/bin" }),
});
const started = await ctrl.start();
out({ ready: true, started, socketPath: ctrl.socketPath, incarnation: ctrl.incarnation, pid: process.pid, binding: ctrl.binding, claim: ctrl.claim ? { claimId: ctrl.claim.claimId, record: ctrl.claim.record } : null });
+27 -4
View File
@@ -2,7 +2,7 @@
// directory: fixture root, project, session file, claim root and socket
// directory. Nothing here touches a live session.
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync } from "node:fs";
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, symlinkSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { spawnSync } from "node:child_process";
@@ -11,15 +11,38 @@ import { ConversationClient } from "../src/client.mjs";
import { AUTHORITY } from "../src/cohort.mjs";
import { FixtureVerifier } from "../src/records.mjs";
import { FakeLauncher } from "./fake-pi.mjs";
import { fixtureDir, piStandIn } from "../../harness/tests/pi-stand-in.mjs";
import { locate } from "../../plugins/harness/pi/binary.mjs";
export const REPO = resolve(import.meta.dirname, "..", "..", "..");
export const SESSION_ID = "0f5e1c2a-1111-4222-8333-944455556666";
export const FAST = Object.freeze({ ack: 1500, state: 1500, start: 1500, clear: 1500, abort: 3000, settle: 3000, grace: 50, write: 1500, maxRounds: 3 });
export const time = (i) => new Date(Date.UTC(2026, 9, 4, 12, 0, 0) + i * 1000).toISOString();
const tmp = mkdtempSync(join(tmpdir(), "chat03-"));
const tmp = fixtureDir("chat03-");
let serial = 0;
// #1873 S4: a stand-in pi package (packages/harness/tests/pi-stand-in.mjs,
// which the fit check passes unless its knobs say otherwise) under `dir`,
// with an npm-style bin link. `found` is the lookup's answer for it, through
// MOSAIC_PI_BIN. The test engine still launches fake-pi.mjs; the stand-in is
// what the controller's fit check and pins read.
export const PI_PACKAGE = "@earendil-works/pi-coding-agent";
export function standInPi(dir, knobs = {}) {
const pkg = join(dir, "prefix", "lib", "node_modules", PI_PACKAGE);
mkdirSync(join(pkg, "dist", "bundle"), { recursive: true });
writeFileSync(join(pkg, "package.json"), JSON.stringify({ name: PI_PACKAGE, version: knobs.version ?? "9.9.9", bin: { pi: "dist/bundle/cli.js" } }));
const cli = join(pkg, "dist", "bundle", "cli.js");
writeFileSync(cli, piStandIn(knobs, { form: knobs.form ?? "node" }), { mode: 0o755 });
mkdirSync(join(dir, "prefix", "bin"));
const bin = join(dir, "prefix", "bin", "pi");
symlinkSync(`../lib/node_modules/${PI_PACKAGE}/dist/bundle/cli.js`, bin);
return { bin, cli, pkg, found: piAt(bin) };
}
export const piAt = (bin) => locate({ MOSAIC_PI_BIN: bin, PATH: "/usr/bin:/bin" });
// The pi every fixture controller checks unless a test passes its own.
export const PI = standInPi(join(tmp, "pi"));
export function cleanupAll() {
spawnSync("chmod", ["-R", "u+rwx", tmp]);
rmSync(tmp, { recursive: true, force: true });
@@ -51,7 +74,7 @@ export const noUnits = Object.freeze({ lookup: async () => ({ state: "absent" })
export function controllerFor(fx, { launcher = new FakeLauncher(), verifier = new FixtureVerifier({ authorities: [AUTHORITY] }), timeouts = FAST, ...opts } = {}) {
const ctrl = new Controller({
fixtureRoot: fx.base, claimRoot: fx.claimRoot, socketDir: fx.socketDir, sessionFile: fx.sessionFile, seat: fx.seat,
launcher, verifier, units: noUnits, timeouts, ...opts,
launcher, verifier, units: noUnits, timeouts, pi: PI.found, ...opts,
});
return { ctrl, launcher, verifier };
}
@@ -118,7 +141,7 @@ export function killChildren() {
// Spawns ctrl-child.mjs. `msgs` collects its JSON lines; `next(pred)` waits
// for one.
export function spawnController(cfg) {
const proc = spawn(process.execPath, [CHILD, JSON.stringify({ timeouts: FAST, ...cfg })], { stdio: ["pipe", "pipe", "pipe"] });
const proc = spawn(process.execPath, [CHILD, JSON.stringify({ timeouts: FAST, piBin: PI.bin, ...cfg })], { stdio: ["pipe", "pipe", "pipe"] });
childPids.add(proc.pid);
const msgs = [];
const waiters = new Set();
+28 -14
View File
@@ -1,8 +1,8 @@
// CHAT-03 (#1507): the pinned Pi binary, started sealed in a scratch home with
// CHAT-03 (#1507): the repository's Pi binary, started sealed in a scratch home with
// no credentials, answers the RPC commands the controller sends with the
// shapes the fake engine models. No prompt is sent, so no model is called.
//
// It also records pinned Pi's startup append (sdk.js 240–252): a session
// It also records Pi 0.85.1's startup append (sdk.js 240–252): a session
// whose branch has no thinking_level_change entry gains one at every start,
// so the leaf moves after launch and the K8 load check fails closed on it.
//
@@ -16,12 +16,20 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import { PassThrough } from "node:stream";
import { LineSplitter, encodeLine } from "../src/framing.mjs";
import { PI_BIN, PI_VERSION, buildPiArgs, checkEnginePin, checkSeal } from "../src/pi-pin.mjs";
import { buildPiArgs, checkSeal } from "../src/pi-seal.mjs";
import { piFit } from "../../harness/src/pi-fit.mjs";
import { locate } from "../../plugins/harness/pi/binary.mjs";
import { levels } from "../../plugins/harness/pi/index.mjs";
import { FakePi } from "./fake-pi.mjs";
import { REPO, assistantEntry, header, thinkingEntry, userEntry } from "./harness.mjs";
import { assistantEntry, cleanupAll, header, thinkingEntry, userEntry } from "./harness.mjs";
// The repository's pi: with no MOSAIC_PI_BIN and no PATH the lookup ends there.
const REPO_PI = locate({});
const scratch = mkdtempSync(join(tmpdir(), "chat03-smoke-"));
after(() => rmSync(scratch, { recursive: true, force: true }));
// harness.mjs makes its fixture directory when it loads; remove it too.
after(cleanupAll);
const COMMANDS = ["get_state", "get_commands", "clear_queue", "abort", "get_tree"];
const exchanges = {};
@@ -90,9 +98,11 @@ async function realPi(tag, sessionFile, { project = "proj", trusted = null, afte
assert.equal(existsSync(join(home, ".pi", "agent", "auth.json")), false);
assert.deepEqual(readdirSync(agent), [], "the agent dir starts empty");
if (trusted) writeFileSync(join(agent, "trust.json"), JSON.stringify({ [realpathSync(join(scratch, trusted))]: true }));
const args = buildPiArgs({ sessionFile });
checkSeal(args);
const pi = spawn(process.execPath, [join(REPO, PI_BIN), ...args, ...after], { cwd: join(scratch, project), env, stdio: ["pipe", "pipe", "pipe"] });
const fit = await piFit(REPO_PI, { levels: levels() });
assert.equal(fit.ok, true, fit.message);
const args = buildPiArgs({ sessionFile, noMcp: fit.noMcp });
checkSeal(args, { noMcp: fit.noMcp });
const pi = spawn(process.execPath, [realpathSync(REPO_PI.bin), ...args, ...after], { cwd: join(scratch, project), env, stdio: ["pipe", "pipe", "pipe"] });
let stderr = "";
pi.stderr.on("data", (c) => (stderr += c));
const exited = new Promise((r) => pi.on("exit", (code, signal) => r({ code, signal })));
@@ -118,11 +128,15 @@ function fakePi(sessionFile) {
const lastId = (file) => JSON.parse(readFileSync(file, "utf8").trim().split("\n").at(-1)).id;
const typesOf = (data) => Object.fromEntries(Object.entries(data ?? {}).map(([k, v]) => [k, v === null ? "null" : Array.isArray(v) ? "array" : typeof v]));
test("the engine pin holds for the installed package", () => {
assert.deepEqual(checkEnginePin(REPO).version, PI_VERSION);
test("the repository's pi passes the fit check and gives a version and a digest", async () => {
assert.equal(REPO_PI.label, "the repository's pi");
const fit = await piFit(REPO_PI, { levels: levels() });
assert.equal(fit.ok, true, fit.message);
assert.equal(fit.version, "0.85.1");
assert.match(fit.digest, /^[0-9a-f]{64}$/);
});
test("pinned Pi, sealed and without credentials, answers the controller's commands with the shapes the fake models", async () => {
test("the repository's Pi, sealed and without credentials, answers the controller's commands with the shapes the fake models", async () => {
const entries = [thinkingEntry(), userEntry("a1b2c3d4", "f0e1d2c3", "hello"), assistantEntry("b2c3d4e5", "a1b2c3d4", "hi")];
const realFile = session("real.jsonl", entries);
const fakeFile = session("fake.jsonl", entries);
@@ -155,7 +169,7 @@ test("pinned Pi, sealed and without credentials, answers the controller's comman
assert.equal(r.command, f.command, type);
assert.equal(r.success, f.success, type);
const rt = typesOf(r.data), ft = typesOf(f.data);
for (const [k, t] of Object.entries(ft)) assert.equal(rt[k], t, `${type}.${k}: the fake models a field pinned Pi doesn't send that way`);
for (const [k, t] of Object.entries(ft)) assert.equal(rt[k], t, `${type}.${k}: the fake models a field the repository's Pi doesn't send that way`);
}
// clear_queue: one queue_update with both queues empty precedes the response, in both.
for (const side of [real, fake]) {
@@ -165,7 +179,7 @@ test("pinned Pi, sealed and without credentials, answers the controller's comman
}
});
test("sealed, pinned Pi ignores a trusted project's .pi resources; --approve past the seal would load them, and checkSeal refuses it (Filbert F2 on #1522)", async () => {
test("sealed Pi ignores a trusted project's .pi resources; --approve past the seal would load them, and checkSeal refuses it (Filbert F2 on #1522)", async () => {
const skill = join(scratch, "trusted", ".pi", "skills", "probe");
mkdirSync(skill, { recursive: true });
writeFileSync(join(skill, "SKILL.md"), "---\nname: probe\ndescription: A project skill that only a trusted load offers.\n---\nprobe\n");
@@ -183,7 +197,7 @@ test("sealed, pinned Pi ignores a trusted project's .pi resources; --approve pas
}
});
test("pinned Pi appends thinking_level_change at start when the branch lacks one, so the leaf moves (K8 then fails closed)", async () => {
test("the repository's Pi appends thinking_level_change at start when the branch lacks one, so the leaf moves (K8 then fails closed)", async () => {
const entries = [userEntry("a1b2c3d4", null, "hello"), assistantEntry("b2c3d4e5", "a1b2c3d4", "hi")];
const realFile = session("bare-real.jsonl", entries);
const fakeFile = session("bare-fake.jsonl", entries);
@@ -197,5 +211,5 @@ test("pinned Pi appends thinking_level_change at start when the branch lacks one
assert.notEqual(side.responses.get_tree.data.leafId, "b2c3d4e5", "the leaf moved off the loaded leaf");
assert.equal(side.responses.get_tree.data.leafId, added[0].id);
}
if (process.env.CHAT03_SMOKE_OUT) writeFileSync(process.env.CHAT03_SMOKE_OUT, JSON.stringify({ pi: PI_VERSION, commands: COMMANDS, exchanges }, null, 2) + "\n");
if (process.env.CHAT03_SMOKE_OUT) writeFileSync(process.env.CHAT03_SMOKE_OUT, JSON.stringify({ pi: (await piFit(REPO_PI, { levels: levels() })).version, commands: COMMANDS, exchanges }, null, 2) + "\n");
});
+49 -6
View File
@@ -5,14 +5,16 @@
import { test, after } from "node:test";
import assert from "node:assert/strict";
import { PassThrough } from "node:stream";
import { realpathSync } from "node:fs";
import { join } from "node:path";
import { Controller, TEST_ENGINE, REPO_ROOT, HANDLED_WITHOUT_RUN, ACK_WITHOUT_START, NO_TURN, RUN_OVERLAP, TRANSPORT_UNKNOWN } from "../src/controller.mjs";
import { AUTHORITY } from "../src/cohort.mjs";
import { FixtureVerifier, sha256 } from "../src/records.mjs";
import { ENGINE_ENV, PI_BIN, SEAL_FLAGS, UNSEALED_ENGINE, checkSeal } from "../src/pi-pin.mjs";
import { ENGINE_ENV, SEAL_FLAGS, UNSEALED_ENGINE, checkSeal } from "../src/pi-seal.mjs";
import { locate } from "../../plugins/harness/pi/binary.mjs";
import { LineSplitter, encodeLine, parseLine } from "../src/framing.mjs";
import { BUSY_ERROR, FakeLauncher, FakePi } from "./fake-pi.mjs";
import { fixture, started, receiptState, outcomeUnknownPush, sessionText, cleanupAll, tick, noUnits, FAST } from "./harness.mjs";
import { fixture, started, receiptState, outcomeUnknownPush, sessionText, cleanupAll, tick, noUnits, FAST, PI, PI_PACKAGE, standInPi } from "./harness.mjs";
after(() => cleanupAll());
@@ -982,12 +984,38 @@ test("N24: the seal is an allow-list: --extension, a missing --no-* flag, a seco
}
});
test("N24c: when --help lists --no-mcp the launch passes it after the seal flags, and the seal then requires it (#1873 S4)", async () => {
const fx = fixture();
const pi = standInPi(join(fx.base, "pi"), { extraFlags: ["--no-mcp"] });
const h = await started({ fx, pi: pi.found });
try {
const args = h.launcher.launches[0].args.slice(1);
assert.deepEqual(args.slice(0, SEAL_FLAGS.length + 5), ["--mode", "rpc", ...SEAL_FLAGS, "--no-mcp", "--session", fx.sessionFile]);
assert.equal(h.ctrl.noMcp, true);
assert.equal(checkSeal(args, { noMcp: true }), true);
const without = args.filter((a) => a !== "--no-mcp");
assert.throws(() => checkSeal(without, { noMcp: true }), (e) => e.code === UNSEALED_ENGINE);
assert.throws(() => checkSeal(args), (e) => e.code === UNSEALED_ENGINE, "--no-mcp where help lacks it");
} finally {
await h.close();
}
// The shared stand-in's help lacks --no-mcp, so its launch has none.
const plain = await started({ fx: fixture() });
try {
assert.equal(plain.launcher.launches[0].args.includes("--no-mcp"), false);
assert.equal(plain.ctrl.noMcp, false);
} finally {
await plain.close();
}
});
test("N24b: the seal covers the engine command and environment: config can't name either, the env is built from names, and a mutated command is refused at bind", async () => {
const fx = fixture();
const make = (extra) => new Controller({ fixtureRoot: fx.base, claimRoot: fx.claimRoot, socketDir: fx.socketDir, sessionFile: fx.sessionFile, seat: fx.seat, launcher: new FakeLauncher(), units: noUnits, timeouts: FAST, ...extra });
// The plain engine option takes extraArgs, cwd and envKeys; the command,
// pre-arguments and environment are the controller's (I3, #1507).
for (const engine of [{ command: "/bin/sh" }, { preArgs: [join(REPO_ROOT, PI_BIN)] }, { command: process.execPath, preArgs: [join(REPO_ROOT, PI_BIN), "--extension", "x"] }, { env: {} }, { env: process.env }, null, [], "pi"]) {
const repoCli = join(REPO_ROOT, "node_modules", PI_PACKAGE, "dist", "bundle", "cli.js");
for (const engine of [{ command: "/bin/sh" }, { preArgs: [repoCli] }, { command: process.execPath, preArgs: [repoCli, "--extension", "x"] }, { env: {} }, { env: process.env }, null, [], "pi"]) {
assert.throws(() => make({ engine }), (e) => e.code === UNSEALED_ENGINE, JSON.stringify(engine));
}
// A configuration is JSON, which can't carry the symbol-keyed test engine.
@@ -1001,7 +1029,7 @@ test("N24b: the seal covers the engine command and environment: config can't nam
assert.throws(() => make({ [TEST_ENGINE]: { command: process.execPath, preArgs: ["fake.mjs", "-e", "x"], env: {} } }), (e) => e.code === UNSEALED_ENGINE);
assert.throws(() => make({ [TEST_ENGINE]: { command: "", preArgs: [], env: {} } }), (e) => e.code === UNSEALED_ENGINE);
// The launch: this Node, the pinned bin, and an environment of named keys
// The launch: this Node, the real file of the pi the lookup found, and an environment of named keys
// only, whatever the controller's own environment holds.
const planted = { NODE_OPTIONS: "--require /tmp/x.cjs", LD_PRELOAD: "/tmp/x.so", PI_PACKAGE_DIR: "/tmp/pkg", ZAI_API_KEY: "zai-test-value", OTHER_API_KEY: "other-test-value" };
const saved = Object.fromEntries(Object.keys(planted).map((k) => [k, process.env[k]]));
@@ -1015,7 +1043,8 @@ test("N24b: the seal covers the engine command and environment: config can't nam
try {
const l = h.launcher.launches[0];
assert.equal(l.command, process.execPath);
assert.equal(l.args[0], join(REPO_ROOT, PI_BIN));
assert.equal(l.args[0], realpathSync(PI.bin));
assert.equal(l.args[0], h.ctrl.engine.preArgs[0]);
assert.equal(l.env.ZAI_API_KEY, "zai-test-value");
for (const k of Object.keys(l.env)) assert.ok(ENGINE_ENV.includes(k) || k === "ZAI_API_KEY", k);
for (const k of ["NODE_OPTIONS", "LD_PRELOAD", "PI_PACKAGE_DIR", "OTHER_API_KEY"]) assert.equal(k in l.env, false, k);
@@ -1027,7 +1056,21 @@ test("N24b: the seal covers the engine command and environment: config can't nam
// A command changed after construction is refused at bind; nothing launches.
const fx2 = fixture();
const launcher = new FakeLauncher();
const ctrl = new Controller({ fixtureRoot: fx2.base, claimRoot: fx2.claimRoot, socketDir: fx2.socketDir, sessionFile: fx2.sessionFile, seat: fx2.seat, launcher, units: noUnits, timeouts: FAST });
// With no pi option the controller takes the one the lookup finds in its own
// environment, read when it is built (#1873 S4): here MOSAIC_PI_BIN names a
// stand-in, which no other step of the lookup would find.
const own = standInPi(join(fx2.base, "env-pi"));
const savedBin = process.env.MOSAIC_PI_BIN;
process.env.MOSAIC_PI_BIN = own.bin;
let ctrl;
try {
ctrl = new Controller({ fixtureRoot: fx2.base, claimRoot: fx2.claimRoot, socketDir: fx2.socketDir, sessionFile: fx2.sessionFile, seat: fx2.seat, launcher, units: noUnits, timeouts: FAST });
assert.deepEqual(ctrl.pi, locate(process.env));
} finally {
if (savedBin === undefined) delete process.env.MOSAIC_PI_BIN; else process.env.MOSAIC_PI_BIN = savedBin;
}
assert.equal(ctrl.pi.label, "MOSAIC_PI_BIN");
assert.deepEqual(ctrl.engine.preArgs, [realpathSync(own.cli)]);
// Closed in finally: a start that wrongly succeeds holds the socket open.
try {
ctrl.engine.command = "/bin/sh";
+6 -1
View File
@@ -30,7 +30,12 @@ scripts/discord-service.sh render | install | uninstall | status <binding>
`--repo` for you; the CLI also takes `--config PATH`.
- `check` validates the binding, the token file (0600, regular, not a
symlink), the context files and the pi install; reads the bot, the guild and
symlink), the context files and the pi the harness lookup finds
(`MOSAIC_PI_BIN`, else the first `pi` on `PATH`, else the repository's;
#1873 S4). `--repo` no longer picks the pi, and the CLI runs no fit check
on it yet, so a pi on `PATH` is used as it is. The service wrapper still
checks that the copy has its own `node_modules/.bin/pi`, but the CLI may run
another pi that the lookup finds first. It then reads the bot, the guild and
every listed channel over REST; opens one gateway connection, waits for
READY, closes it. Nothing is sent to a channel. Close code 4014 is reported
as the message-content intent not being granted in the developer portal.
+7 -3
View File
@@ -46,7 +46,7 @@
// 3 refused by a brake (STOP present or the binding held; a supervisor must
// not retry); 4 usage.
import { existsSync, mkdirSync, mkdtempSync, writeFileSync } from "node:fs";
import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { createHash } from "node:crypto";
import { DiscordError, printable } from "./errors.mjs";
@@ -57,6 +57,7 @@ import { defaultConfigPath, loadDataRoot, bindingPath, bindingDataDir, loadBindi
import { createRest } from "./rest.mjs";
import { createGateway, CONNECTOR_INTENTS } from "./gateway.mjs";
import { createEngine, buildPiArgs, sessionContinues } from "./engine-pi.mjs";
import { locate } from "../../plugins/harness/pi/binary.mjs";
import { TOOLS_ENV, enabledToolNames } from "./tools.mjs";
import { createSetsparkApi } from "./setspark.mjs";
import { assembleContext } from "./context.mjs";
@@ -134,8 +135,11 @@ function prepare(opts) {
checkLibraryEngine(dirname(opts.config), binding);
const contextFiles = resolveContextFiles(binding, opts.repo);
const toolRoots = resolveToolRoots(binding, { dataRoot });
const pi = join(opts.repo, "node_modules", ".bin", "pi");
if (!existsSync(pi)) throw new DiscordError("pi not found at node_modules/.bin/pi in the repository; run npm ci in the repository");
// #1873 S4: the pi the harness lookup finds (MOSAIC_PI_BIN, the first pi on PATH, the
// repository's), not only the repository's copy.
const found = locate(process.env);
if (!found.bin) throw new DiscordError(found.problem);
const pi = found.bin;
const journalDir = bindingDataDir(dataRoot, binding.name);
const sessionDir = join(dataRoot, "sessions", `discord-${binding.name}`);
return { dataRoot, bindingFile, binding, contextFiles, toolRoots, pi, journalDir, sessionDir };
+10 -3
View File
@@ -1,5 +1,5 @@
// Shared fakes for the offline suite. No network, no token, no model.
import { mkdtempSync, mkdirSync, writeFileSync, rmSync, chmodSync } from "node:fs";
import { mkdtempSync, mkdirSync, writeFileSync, rmSync, chmodSync, symlinkSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { after } from "node:test";
@@ -53,12 +53,19 @@ export function binding(overrides = {}) {
return validateBinding(rawBinding(overrides));
}
// A fake repository with the files prepare() looks for.
// A fake repository with the files prepare() looks for. Its pi is a package the
// harness lookup accepts, and MOSAIC_PI_BIN names it for every CLI this test
// process starts (#1873 S4), so no test reads the host's own pi.
export function makeRepo(root) {
const repo = join(root, "repo");
const pkg = join(repo, "node_modules", "@earendil-works", "pi-coding-agent");
mkdirSync(join(repo, "node_modules", ".bin"), { recursive: true });
mkdirSync(join(pkg, "dist"), { recursive: true });
mkdirSync(join(repo, "contracts"), { recursive: true });
writeFileSync(join(repo, "node_modules", ".bin", "pi"), "#!/bin/sh\nexit 0\n", { mode: 0o755 });
writeFileSync(join(pkg, "package.json"), JSON.stringify({ name: "@earendil-works/pi-coding-agent", version: "0.0.0-test", bin: { pi: "dist/cli.js" } }));
writeFileSync(join(pkg, "dist", "cli.js"), "#!/bin/sh\nexit 0\n", { mode: 0o755 });
symlinkSync("../@earendil-works/pi-coding-agent/dist/cli.js", join(repo, "node_modules", ".bin", "pi"));
process.env.MOSAIC_PI_BIN = join(repo, "node_modules", ".bin", "pi");
writeFileSync(join(repo, "contracts", "CONSTITUTION.md"), "# constitution\nbe good\n");
return repo;
}
+14 -3
View File
@@ -1,19 +1,21 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import {spawnSync} from 'node:child_process';
import {writeFileSync,chmodSync,unlinkSync,existsSync} from 'node:fs';
import {join} from 'node:path';
import {writeFileSync,chmodSync,unlinkSync,existsSync,mkdirSync,symlinkSync} from 'node:fs';
import {dirname,join} from 'node:path';
import {fileURLToPath} from 'node:url';
import {makeRoot,makeRepo,makeDeployment} from './helpers.mjs';
const CLI=fileURLToPath(new URL('../src/cli.mjs',import.meta.url));
function setup(){
const root=makeRoot(),repo=makeRepo(root),d=makeDeployment(root,{}, {tokenContent:'MTAw.abcdefghijklmnopqrstuvwxyz0123456789'}),calls=join(root,'network-called'),preload=join(root,'no-network.mjs');
writeFileSync(preload,`import {writeFileSync} from 'node:fs';const refuse=()=>{writeFileSync(${JSON.stringify(calls)},'called');throw Error('fixture refuses network')};globalThis.fetch=refuse;globalThis.WebSocket=class{constructor(){refuse()}};`);
const run=(command='check',args=[])=>spawnSync(process.execPath,['--import',preload,CLI,command,d.raw.name,'--config',d.config,'--repo',repo,...args],{encoding:'utf8',timeout:5000});
const run=(command='check',args=[],env=process.env)=>spawnSync(process.execPath,['--import',preload,CLI,command,d.raw.name,'--config',d.config,'--repo',repo,...args],{encoding:'utf8',timeout:5000,env});
return {root,repo,d,calls,run};
}
test('check --offline validates local fixture without REST or gateway calls',()=>{
const s=setup(),r=s.run('check',['--offline']);assert.equal(r.status,0,r.stderr);assert.match(r.stdout,/token file mode 0600 ok; STOP absent/);assert.match(r.stdout,/offline check passed; no REST or gateway calls were made\n$/);assert.ok(!existsSync(s.calls));
// #1873 S4: the fixture's own pi, which the helpers name in MOSAIC_PI_BIN, never the host's.
assert.ok(r.stdout.includes(`pi ${join(s.repo,'node_modules','.bin','pi')};`),r.stdout);
// Online check still reaches REST: the trap fails it immediately.
const online=s.run();assert.equal(online.status,1);assert.ok(existsSync(s.calls));assert.match(online.stderr,/fixture refuses network/);
});
@@ -24,3 +26,12 @@ test('check --offline refuses token mode and missing context under the selected
test('run --offline is a usage refusal before reading config or networking',()=>{
const s=setup();unlinkSync(s.d.config);const r=s.run('run',['--offline']);assert.equal(r.status,4,r.stderr);assert.match(r.stderr,/--offline/);assert.ok(!existsSync(s.calls));
});
test('#1873 S4: check takes the pi the harness lookup finds; MOSAIC_PI_BIN names it, and a bad one refuses with the lookup line, no path',()=>{
// A pi package the way npm installs one under a prefix: its bin is linked from prefix/bin.
const s=setup(),pkg=join(s.root,'prefix','lib','node_modules','@earendil-works','pi-coding-agent'),bin=join(s.root,'prefix','bin','pi');
mkdirSync(join(pkg,'dist'),{recursive:true});mkdirSync(dirname(bin),{recursive:true});
writeFileSync(join(pkg,'package.json'),JSON.stringify({name:'@earendil-works/pi-coding-agent',version:'9.9.9',bin:{pi:'dist/cli.js'}}));
writeFileSync(join(pkg,'dist','cli.js'),'#!/bin/sh\nexit 0\n',{mode:0o755});symlinkSync(join(pkg,'dist','cli.js'),bin);
const named=s.run('check',['--offline'],{...process.env,MOSAIC_PI_BIN:bin});assert.equal(named.status,0,named.stderr);assert.ok(named.stdout.includes(`pi ${bin};`),named.stdout);
const bad=s.run('check',['--offline'],{...process.env,MOSAIC_PI_BIN:'relative/pi'});assert.equal(bad.status,2,bad.stderr);assert.match(bad.stderr,/MOSAIC_PI_BIN isn't an executable file at an absolute path/);assert.ok(!bad.stderr.includes(s.root));assert.ok(!existsSync(s.calls));
});
+149 -7
View File
@@ -18,13 +18,155 @@ Every harness layer below is a row S0 line
(`docs/plans/2026-10-04_slice-1.md`, "What S6 can rely on"), and each
bundle's `manifest.json` names the lines it relies on (`reliesOn`):
| | Pi | Claude Code |
|---|---|---|
| 1. block a tool | `tool_call` handler returns a block | PreToolUse command hook exits 2; `--bare` is never passed |
| 2. gate crash | a throw in the handler blocks | the hook runs as `<gate> \|\| exit 2` |
| 3. gate missing | a missing `-e` file refuses to start | the wrapper blocks |
| 4. gate hang | the runner's wall clock plus the `agent_end` marker | `timeout -k 2 10 <gate> \|\| exit 2`, hook timeout 20 |
| 5. bash | limited only by the tool limit | limited only by the tool limit |
| | Pi | Claude Code | How pi's line is checked |
|---|---|---|---|
| 1. block a tool | `tool_call` handler returns a block | PreToolUse command hook exits 2; `--bare` is never passed | at first sight of a binary, by the fit check's model run, on the pi that runs; and suites (`pi-session.test.mjs`) |
| 2. gate crash | a throw in the handler blocks | the hook runs as `<gate> \|\| exit 2` | at first sight of a binary, by the fit check's model run; and suites |
| 3. gate missing | a missing `-e` file refuses to start | the wrapper blocks | at every start, by the fit check, on the pi that runs; and suites |
| 4. gate hang | the runner's wall clock plus the `agent_end` marker | `timeout -k 2 10 <gate> \|\| exit 2`, hook timeout 20 | the `agent_end` marker: at first sight of a binary, by the fit check's model run; and suites |
| 5. bash | limited only by the tool limit | limited only by the tool limit | suites, on the repository's pi |
Pi is not pinned (#1873 pi-unpin). Before a session starts, `preflight`
awaits the fit check in `src/pi-fit.mjs` on the binary the module found
(the Triage host and route await the module's `ready` instead), and
`env` reads its pass. It runs that binary in a sandbox with the network
unshared and calls no real model. Each run is a child process the check
awaits, up to four at a time, so the bus's event loop keeps running
while it does; callers that ask while a check runs share it. Each run
gets the environment a bus session's pi gets, built by the same code
(the launcher's `ENV_ALLOW` and `sessionEnv`, the sandbox's
`sessionSetenv`, the runner's `adapterEnv` and the pi module's
`harnessEnv`) from the bus process's own environment, less the provider
keys, and starts the binary through `/bin/sh` as the adapter does. So
`HOME` is a scratch home, `TMPDIR` is `/tmp` bound to scratch, a run has
`TERM`, `TZ` or `LC_ALL` only when the bus has it (a bus started by its
unit has none), and no run sets `PI_CODING_AGENT_DIR`, which no session
has: pi's agent directory is `$HOME/.pi/agent`, where the user sources
are planted. A test fails when the fit runs' names, or the values the
code sets, differ from a bus session's, apart from the provider
variables, for a login shell's host and a unit's. It needs
`--help` to list, in its option column, every flag the adapter passes
(with rpc mode and each thinking level), and a missing `-e` file to stop
the start in rpc and in `-p`. The planted ambient sources must stay
unloaded with the seal flags on (skills and prompt templates are read
from `get_commands` in rpc and from the `-e` extension's command list in
`-p`): extensions, MCP
servers, skills and prompt templates (found in their directories, listed
in a `settings.json`, or in a local package a `settings.json` lists),
context files (`AGENTS.md`, `CLAUDE.md`, `SYSTEM.md`) and a trusted
project's `.pi/SYSTEM.md` and `.pi/APPEND_SYSTEM.md`. The project has a
saved trust, so only `--no-approve` keeps those two out. An `-e`
extension writes the system prompt, command names and session id it sees
at `session_start`; the sealed runs must show the `--system-prompt` file
and none of the planted sources. A control run passes the seal flags less every loader flag
(`--no-approve`, `--no-extensions`, `--no-skills`,
`--no-prompt-templates`, `--no-context-files`, and `--no-mcp` when
listed), so it keeps `--offline`, `--no-themes`, `--no-tools` and
`--no-session`, and the same `-e` and `--system-prompt`. It must show
each kind of source (an MCP server only when `--help` lists `--no-mcp`)
and answer `get_state` and `get_commands`, and its `-e` extension must
list the planted skill and prompt template, or the check can't tell and
refuses. For every kind but the trusted project files, only a source
outside the project counts there, since a project source can wait on
trust and `--no-approve` would then hide a seal flag the binary ignores.
Then one sealed rpc run per loader flag leaves out only that flag: its
kind must show, and no other kind (an MCP server may show beside an
extension when `--help` lists no `--no-mcp`, since pi then ties MCP to
`--no-extensions`). If `--no-extensions` also disables the MCP loader,
the check leaves out both it and `--no-mcp` in one additional rpc run.
That run must show MCP and no kind beyond MCP and extensions. Together
with the run leaving out only `--no-extensions` (extensions load, MCP
does not), this proves `--no-mcp` with its loader enabled. The planted
MCP servers have distinct names so a trusted project entry cannot hide
the user entry. No version comparison selects these checks.
So each loader flag is shown to keep its own kind out, and no other
seal flag, nor their order, can stand in for it. Five
sealed `-p` runs have a bus session's shape: the arguments the adapter
passes for a session, in its order, from `sessionShape` in
`src/pi-fit.mjs`, with two `-e` and `--session-dir`. The adapter's four
session branches (a tool list or `--no-tools`, `-c` over a stored
session or none, `--thinking` or none, `--mode json` or none) are spread
over the five runs so every pair of them is passed together. A `-c` run
gets a stored session and must resume it. Each must load both `-e`
files and show no planted source. A test drives the adapter, in a bus
session's environment, through every branch set with no tools, one or
two, `-c` over a pi-named session file or an unrelated file, each
thinking level and each provider a session can name. It runs over a
login shell's host and a unit's, and fails on an argv `sessionShape`
doesn't build, a pair no run passes, or a variable the adapter reads
that no session sets. So a branch on `TZ` or `LC_ALL` being unset, or on
`TERM` missing from the environment, turns it red. (Bash run as
`/bin/sh` sets an unset `TERM` to `dumb` for the shell without exporting
it, so `${TERM:-}` in the adapter is never empty, in a session or here.)
It can't see a branch keyed on a
value it doesn't try (a model, a path, another tool or provider). A last `-p` run is the
argv the adapter builds in `-p` with `MOSAIC_SKILLS`, one `-e` file, no
tools and no session (`workerShape`), so `--no-skills` and then two
`--skill` directories, `--no-tools`, `--no-session`, `--provider` and
`--model`. A test runs the adapter and holds its argv to `workerShape`.
No caller makes this exact run today: task launches refuse skill
directories, and only the interactive launch (`scripts/agent.sh`) sets
`MOSAIC_SKILLS`, in the TUI, which no fit run can drive. It is the
nearest headless form of the adapter's `--skill` argv.
The run's `-e` extension must list both skills and no planted one. pi
0.85.1 adds a `--skill` directory to discovery, so a pi that let
`--skill` bring the planted skills back is refused there, and so is one
that drops a `--skill` directory once `--no-skills` is on. This run
replaces the plain sealed `-p` run with `--no-session` and no `--skill`,
so it adds no run. Each run gets a fresh scratch directory, so no run sees
what another left. Not run: `--fork` and the interactive TUI
(worker containers and `scripts/agent.sh` only), `-p` with
`--no-session` and no `--skill` (a task launch's shape; every bus session
passes `--session-dir`), `--skill` beside `--tools`, `--session-dir`,
`-c`, `--thinking` or `--mode json`, or in rpc (the TUI launch passes the
first three; no bus session passes `--skill`), `--skill` with no `-e`
(every fit run needs its `-e` extension to see what loaded), one or three
or more `--skill` directories, the branches three or
four at a time beyond the five runs, `-c` over a session directory that
holds nothing for the project (one more run costs about 340 ms cold),
and a session's own values (tools beyond `read` and `bash`, thinking
levels beyond `low`, provider, model, key, `-e` files, request). The
runs are offline with no key, so nothing past the start is tried. A pi
built to act one way in the fit runs and another way with those is
outside what a start-time check can catch. A run killed for taking
over 20 s or printing over 4 MB can't be checked and says which. A
`--version` that fails or hangs past 3 s only leaves no label. Pi
0.85.1 appends the agent directory's `APPEND_SYSTEM.md` whatever the
flags, so the check doesn't plant one. Any failure refuses the launch with
`harness-feature-missing` and one line that names the binary by its label,
for example `pi 2.0.0 (MOSAIC_PI_BIN) lacks --no-context-files`. When
`--help` lists `--no-mcp`, the adapter passes it too. A pass is cached in
memory by a digest of the package's files (and npm's lock files, for
the repository's copy) and the names in the runs' environment, so a bus
restarted under other names checks again.
One more run (#1873 S6) talks to a scripted model, so lines 1, 2 and 4
and the system prompt are checked on the pi that runs. It is a
session-shaped `-p` run with `--tools read`, the gate's `-e` extension
and a second one, `src/pi-fit-throw.mjs`, on a stand-in provider and
model named `fit-model` (never a Claude model). The model is
`src/pi-fit-model.mjs`: inside the sandbox it is the first process, it
listens on the sandbox's own loopback (the network stays unshared),
writes the agent directory's `models.json` for its port, starts pi and
answers one scripted tool call per request. The script reads a project
file the gate allows, a file outside the project, the same outside file
through a link under each of the four spellings pi 0.85.1 tries for a
missing path (a narrow space before AM, decomposed accents, a curly
apostrophe, both), and a project file the second extension's handler
throws for. The run passes only when the allowed read returns the file,
every blocked read is a tool error carrying the gate's own reason and
no request carries the outside file's word, the throw stops its read,
every request carries the `--system-prompt` file, the script runs to
its end, and the gate's `agent_end` hook writes the turn marker with the
model's final text answer. A pi that runs a blocked tool, swallows a
handler's throw, drops the system prompt or never emits `agent_end` is
refused. A run that times out, exits non-zero, or whose model gets no
request or sees the run end early, can't be checked and refuses as
undecided. A run that leaked and then hung is refused, not undecided. Its verdict is part of the cached pass, so it
runs once per digest. It runs beside the other runs, so a cold check
takes about as long as before. Line
5 (bash) is not checked at start, and nor is a read spelling pi may add
later; a stand-in can only check what it scripts.
Claude Code also runs with `--restricted`: no user, project or local
settings, file tools confined to the working directory, and no `CLAUDE.md`
+8 -3
View File
@@ -100,7 +100,8 @@ export function sessionModel(resolved, models = null, which = "primary", { proxy
if (models === null) return { harness, provider: null, model, thinking };
const entry = Object.hasOwn(models, model) ? models[model] : null;
if (!entry) throw new BundleError(`instance ${resolved.instance}: unknown model table selection for ${ROUTE_VARS[which][1]} var`);
const transport = entry.provider === "proxy" || proxy ? checkedTransport(entry, null, resolver) : null;
// #1873 harness-matrix: the route's own harness, so a model the module holds off it refuses here.
const transport = entry.provider === "proxy" || proxy ? checkedTransport(entry, harness, resolver) : null;
return { harness, provider: entry.provider, model: entry.model, modelKey: model, thinking,
...(transport ? { proxyRoute: transport.routeId, proxyTransport: transport } : {}) };
@@ -432,9 +433,13 @@ export function buildBundle({ dir, resolved, contract, arbiter = false, tracker
];
// The module's own files. A name the bundle already wrote refuses at write (flag wx).
const transports = {};
// #1873 harness-matrix: only models the provider's route() lets this session's harness run, so a
// pi session's models.json never names an upstream claude model or a Claude-named legacy entry.
// route() has no account data: a module entry the catalogue holds to claude-code for an OAuth name
// can still be listed. Only the session's own model is admitted against the catalogue.
if (proxy) for (const [key, entry] of Object.entries(models ?? {})) {
try { transports[key] = checkedTransport(entry, null, resolver); }
catch (e) { if (e?.code !== 'proxy-provider-unsupported') throw e; }
try { transports[key] = checkedTransport(entry, model.harness, resolver); }
catch (e) { if (!['proxy-provider-unsupported', 'model-harness-incompatible'].includes(e?.code)) throw e; }
}
const own = harness.bundle({ dir, files: { ...files }, session: { ...session, ...model }, write, quote, models, proxy, transports });
Object.assign(files, own.files);
+153
View File
@@ -1,2 +1,155 @@
// Shared by the sender and recipient's reply-cycle check.
export const LENGTH_FAILURE_NOTICE_PREFIX = 'Mosaic turn failure: ';
// #1873 chat-fail F1: the plain cause of a failed turn. It lives here, not in a file of its own,
// because the session sandbox mounts only RUNNER_FILES (packages/seat/src/sandbox.mjs) and this
// file is already one of them.
//
// The adapter contract (adapters/README.md) puts the answer on stdout and makes a
// nonzero exit a failure. A harness that ends on an error result may still print that
// result on stdout before it exits nonzero. That text is the only cause the runner
// has, so the chat line carries it, made safe first:
//
// 0. %XX escapes and HTML character references are decoded and the text is put in NFKC
// form, so an encoded or full-width email, path or secret is seen as itself.
// 1. ANSI escapes go, every control, format, separator, surrogate and private-use
// character goes, and whitespace collapses to single spaces.
// 2. If a secret the turn's environment holds (a variable named like a key, token,
// secret, password or credential) appears in the text, even split by whitespace or
// controls, no text is shown at all.
// 3. Links, email addresses, absolute and Windows paths, "Bearer x" values, the value after
// a key word ("key: x", "\"api_key\":\"x\"", "token x"), known key prefixes and long
// letter-and-digit runs become [redacted].
// 4. The rest is cut to CAUSE_MAX characters.
//
// Only the last READ_MAX characters are classified and shown. A harness prints its error
// result last, and an answer streamed before the failure may itself talk about limits or
// signing in. Redaction runs over the whole text first, so the cut never parts a key word
// from its value.
//
// Three causes are recognised from the text and shown as a fixed plain line with the
// next step: the account is out of usage, its sign-in expired or was refused, or it
// doesn't offer the model. Anything else shows the cleaned text. The fixed lines never
// carry any of the error's own words.
export const CAUSE_MAX = 300;
const READ_MAX = 2000;
export const PLAIN = Object.freeze({
usage: "the model account is out of usage. Check Settings > Model setup.",
signin: "the model account's sign-in expired or was refused. Sign in again in Settings > Model setup.",
model: "the model this agent uses isn't offered by its account. Pick another model in Settings > Model setup.",
});
export const WITHHELD = "the error text held a secret, so it isn't shown. Check Settings > Model setup.";
const ANSI = /\u001b(?:\[[0-?]*[ -/]*[@-~]|\][^\u0007\u001b]*(?:\u0007|\u001b\\)?|[@-Z\\-_])/g;
// Cf (format: zero-width, bidi overrides) is removed so a split token rejoins and is seen.
const INVISIBLE = /[\p{Cf}\p{Cs}\p{Co}]/gu;
// Cc (C0, DEL, C1), line and paragraph separators become spaces.
const BREAKS = /[\p{Cc}\p{Zl}\p{Zp}]/gu;
const USAGE = /usage limit|hit your [\w -]{0,24}limit|(?<!context |output |token |length )limit reached|credit balance is too low|insufficient[_ ]quota|quota (?:exceeded|exhausted)|exceeded your current quota|out of (?:usage|credits)/i;
const SIGNIN = /(?:oauth |access |auth )?token (?:has )?expired|invalid api key|invalid x-api-key|invalid bearer token|authentication_error|authentication failed|run \/login|not logged in|unauthori[sz]ed|(?:error|status)[: ]+401\b|sign[- ]?in (?:again|required|expired)|log ?in (?:again|required|expired)|invalid_grant/i;
const MODEL = /issue with the selected model|model_not_found|\bmodel\b.{0,80}\b(?:not found|does not exist|not available|isn't available|not supported|may not exist|not offered|no access)|not_found_error.{0,80}\bmodel\b|unknown model|no such model/i;
const ENTITY = { amp: "&", lt: "<", gt: ">", quot: '"', apos: "'", commat: "@", lowbar: "_", period: ".", colon: ":", sol: "/", bsol: "\\", num: "#", equals: "=", hyphen: "-", dash: "-" };
function codePoint(n) {
return Number.isInteger(n) && n > 0 && n <= 0x10ffff && (n < 0xd800 || n > 0xdfff) ? String.fromCodePoint(n) : " ";
}
// %XX escapes (twice encoded included) and HTML character references are decoded, then NFKC
// folds full-width and compatibility forms to their plain letters.
export function decodeText(text) {
let out = String(text);
for (let i = 0; i < 3; i++) {
const next = out
.replace(/(?:%[0-9a-f]{2})+/gi, run => { try { return decodeURIComponent(run); } catch { return run.replace(/%([0-9a-f]{2})/gi, (_, h) => String.fromCharCode(parseInt(h, 16))); } })
.replace(/&#(\d{1,7});?|&#x([0-9a-f]{1,6});?|&([a-z]{2,7});/gi, (all, dec, hex, name) =>
dec ? codePoint(Number(dec)) : hex ? codePoint(parseInt(hex, 16)) : ENTITY[name.toLowerCase()] ?? all);
if (next === out) break;
out = next;
}
return out.normalize("NFKC");
}
// The text with every space, control, format, mark, surrogate and private-use character gone,
// for the env-secret check.
function compactText(text) {
return decodeText(text).replace(ANSI, "").replace(/[\p{Cc}\p{Cf}\p{Cs}\p{Co}\p{M}\p{Z}\s]/gu, "");
}
const SECRET_NAME = /KEY|TOKEN|SECRET|PASS|CRED|AUTH|COOKIE/i;
// Values from the turn's environment that must never reach a chat line.
export function secretValues(env) {
const out = [];
for (const [name, value] of Object.entries(env ?? {})) {
if (typeof value !== "string" || !SECRET_NAME.test(name)) continue;
const compact = compactText(value);
if (compact.length >= 8) out.push(compact);
}
return out;
}
// A key word, with any prefix joined by _ or - (access_token, client_secret, x-api-key).
const KEY_WORD = "(?:\\w*[_-])?(?:api[_-]?key|key|token|secret|password|passwd|authorization|cookie|session)";
// Redaction runs over the whole text, up to the runner's 128000 characters, so a pattern that
// could start anywhere inside a long run without spaces starts only where the run starts: it
// matches the same text in linear time, not quadratic.
const REDACT = [
/(?<![a-z0-9+.-])[a-z0-9+.-]+:\/\/\S*/gi, // any link, its userinfo and query included, from the run's start
/\bwww\.\S+/gi,
/(?<!\S)(?=\S*\S@\S)\S+/g, // email addresses and user@host: the whole run holding the @
/(?:^|(?<=[\s("'=:]))(?:~|\.{1,2})?\/\S+/g, // absolute and home paths
/\b[A-Za-z]:\\\S*/g, // Windows paths
/\\\\\S+/g, // UNC paths
/\b(?:bearer|basic)\s+\S+/gi,
// The value after a key word: "key: x", "key=x" and the JSON form "\"api_key\":\"x\"" always;
// "token x" with only a space when x is quoted, holds a digit or is 8 characters or more.
new RegExp(`\\b${KEY_WORD}["']?\\s*[:=]\\s*(?:"[^"]*"?|'[^']*'?|[^\\s,;}]+)`, "gi"),
new RegExp(`\\b${KEY_WORD}["']?\\s+(?:"[^"]*"?|'[^']*'?|(?=[^\\s"',;}]*\\d)[^\\s"',;}]+|[^\\s"',;}]{8,})`, "gi"),
/\b(?:sk|msk|rk|pk|ghp|gho|ghs|glpat|xox[a-z])[-_]\S+/gi,
/\beyJ\S+/g, // JWT
// A long run that mixes letters and digits, or any very long run, reads as a key.
/[A-Za-z0-9_\-+/=.]{32,}/g,
/(?<![A-Za-z0-9_\-+/=.])(?=[A-Za-z0-9_\-+/=.]*\d)(?=[A-Za-z0-9_\-+/=.]*[A-Za-z])[A-Za-z0-9_\-+/=.]{16,}/g,
];
export function cleanText(text) {
return String(text).replace(ANSI, "").replace(INVISIBLE, "").replace(BREAKS, " ").replace(/\s+/g, " ").trim();
}
// A text read from its tail keeps its end, where the error is; any other keeps its start.
function cap(text, fromEnd = false, n = CAUSE_MAX) {
const chars = Array.from(text);
if (chars.length <= n) return text;
return fromEnd ? "..." + chars.slice(-(n - 3)).join("") : chars.slice(0, n - 3).join("") + "...";
}
// The plain cause for a failed turn's stdout, or null when there is no text.
// `secrets` is secretValues(env) for the turn.
export function turnCause(stdout, secrets = []) {
if (typeof stdout !== "string") return null;
const decoded = decodeText(stdout);
const clean = cleanText(decoded);
if (!clean) return null;
// The secret check covers the whole text, not only the part that is read.
const compact = compactText(stdout);
if (secrets.some(value => compact.includes(value))) return WITHHELD;
// Only the tail is classified and shown. Redaction runs over the whole text before any cut,
// so a cut that lands between a key word and its value can't leave the value bare.
const tail = clean.length > READ_MAX;
const read = tail ? clean.slice(-READ_MAX) : clean;
if (USAGE.test(read)) return PLAIN.usage;
if (SIGNIN.test(read)) return PLAIN.signin;
if (MODEL.test(read)) return PLAIN.model;
let shown = clean;
for (const re of REDACT) shown = shown.replace(re, "[redacted]");
shown = shown.replace(/(?:\[redacted\]\s*){2,}/g, "[redacted] ").trim();
// The cut drops the partial word it lands in; a redacted text short enough needs no cut.
if (shown.length > READ_MAX) shown = shown.slice(-READ_MAX).replace(/^\S+\s*/, "");
// Text that redacts to nothing but [redacted] says nothing; the technical reason stands alone.
return shown && shown !== "[redacted]" ? cap(shown, tail) : null;
}
// The chat body for a failed turn: the plain cause first, the technical reason after.
export const DETAILS = "\nDetails: ";
export function failureBody(turn) {
return turn.cause ? `The turn failed: ${turn.cause}${DETAILS}${turn.reason}` : `The turn failed: ${turn.reason}`;
}
+111
View File
@@ -0,0 +1,111 @@
// #1873 pi-unpin S6: the scripted model the fit check's model run talks to (pi-fit.mjs). It
// answers the Messages API one scripted step at a time: step k is a tool call, given when the
// request carries k tool results; after the last step it answers with fixed text. Only node's own
// modules are imported, since the file runs inside the fit check's sandbox.
//
// Run as a program, inside the sandbox: `node pi-fit-model.mjs <config.json> -- <command...>`. It
// listens on 127.0.0.1 port 0 in the sandbox's own network namespace (checkArgv unshares the
// network, so only that namespace's loopback reaches it), writes the agent directory's models.json
// naming that port for the stand-in provider, starts the command with its own environment and
// stdio, writes what each request held to the record file after each request, and exits with the
// command's status. It is the first process in the sandbox's pid namespace, so when it ends, or is
// killed, every process in the sandbox ends with it. No real model and no host network are
// reached: the sandbox has no other network.
//
// packages/harness/tests/helpers.mjs mockAnthropic serves the same answers to session tests.
import { createServer } from "node:http";
import { spawn } from "node:child_process";
import { mkdirSync, readFileSync, realpathSync, writeFileSync } from "node:fs";
import { dirname } from "node:path";
import { fileURLToPath } from "node:url";
export const textOf = (c) => (typeof c === "string" ? c : Array.isArray(c) ? c.map((b) => b?.text ?? "").join("") : "");
// The tool results in a request's messages, in order: { id, isError, content }.
export function toolResults(messages) {
const out = [];
for (const m of messages ?? []) {
if (m?.role !== "user" || !Array.isArray(m.content)) continue;
for (const b of m.content) if (b?.type === "tool_result") out.push({ id: b.tool_use_id, isError: b.is_error ?? false, content: textOf(b.content) });
}
return out;
}
// A request handler for the Messages API. script: [{ name, input }], one tool call per step.
// done(results): the text that ends the run once the script is spent. requests: each request is
// pushed as { tools, results, system, answer, raw }. onRequest: called after each push. A request
// with no tools gets "mock". Streaming and plain answers both work; count_tokens answers 10.
export function messagesHandler({ script = [], done = (r) => `DONE ${JSON.stringify(r)}`, requests = [], onRequest = () => {} } = {}) {
return (req, res) => {
let raw = "";
req.on("data", (c) => (raw += c));
req.on("end", () => {
if (req.method !== "POST" || !/\/messages(\?|$)/.test(req.url)) {
res.writeHead(req.url.includes("count_tokens") ? 200 : 404, { "content-type": "application/json" });
return res.end(JSON.stringify(req.url.includes("count_tokens") ? { input_tokens: 10 } : { type: "error", error: { type: "not_found_error", message: "mock" } }));
}
let body;
try { body = JSON.parse(raw || "{}"); } catch { body = {}; }
const tools = Array.isArray(body.tools) ? body.tools.map((x) => x.name) : [];
const results = toolResults(body.messages);
const step = tools.length ? script[results.length] : null;
const r = step ? { kind: "tool", id: `toolu_${results.length + 1}`, name: step.name, input: step.input } : { kind: "text", text: tools.length ? done(results) : "mock" };
requests.push({ tools, results, system: body.system, answer: r, raw });
onRequest();
const content = r.kind === "tool" ? { type: "tool_use", id: r.id, name: r.name, input: {} } : { type: "text", text: "" };
const delta = r.kind === "tool" ? { type: "input_json_delta", partial_json: JSON.stringify(r.input) } : { type: "text_delta", text: r.text };
const stop = r.kind === "tool" ? "tool_use" : "end_turn";
if (!body.stream) {
res.writeHead(200, { "content-type": "application/json" });
const c = r.kind === "tool" ? { type: "tool_use", id: r.id, name: r.name, input: r.input } : { type: "text", text: r.text };
return res.end(JSON.stringify({ id: "msg_mock", type: "message", role: "assistant", model: body.model, content: [c], stop_reason: stop, stop_sequence: null, usage: { input_tokens: 10, output_tokens: 5 } }));
}
const send = (event, data) => res.write(`event: ${event}\ndata: ${JSON.stringify({ type: event, ...data })}\n\n`);
res.writeHead(200, { "content-type": "text/event-stream", "cache-control": "no-cache" });
send("message_start", { message: { id: "msg_mock", type: "message", role: "assistant", model: body.model, content: [], stop_reason: null, stop_sequence: null, usage: { input_tokens: 10, output_tokens: 1, cache_creation_input_tokens: 0, cache_read_input_tokens: 0 } } });
send("content_block_start", { index: 0, content_block: content });
send("content_block_delta", { index: 0, delta });
send("content_block_stop", { index: 0 });
send("message_delta", { delta: { stop_reason: stop, stop_sequence: null }, usage: { output_tokens: 5 } });
send("message_stop", {});
res.end();
});
};
}
// How much of each tool result the record keeps.
const KEEP = 2000;
// config: { record, models, provider, model, key, script, done, secret }. record: the file each
// request is written to, as { requests: [{ tools, system, results: [{ isError, content }],
// done, leak }] }: tools is the count of tools offered, done whether the answer was the done text,
// leak whether the raw request held the secret word.
async function main(argv) {
const [config, sep, ...command] = argv;
if (!config || sep !== "--" || !command.length) process.exit(2);
const c = JSON.parse(readFileSync(config, "utf8"));
const requests = [];
const record = () => writeFileSync(c.record, JSON.stringify({ requests: requests.map((r) => ({
tools: r.tools.length,
system: textOf(r.system),
results: r.results.map((x) => ({ isError: Boolean(x.isError), content: String(x.content).slice(0, KEEP) })),
done: r.answer.kind === "text" && r.answer.text === c.done,
leak: r.raw.includes(c.secret),
})) }));
const server = createServer(messagesHandler({ script: c.script, done: () => c.done, requests, onRequest: record }));
await new Promise((resolve, reject) => { server.once("error", reject); server.listen(0, "127.0.0.1", resolve); });
mkdirSync(dirname(c.models), { recursive: true });
writeFileSync(c.models, JSON.stringify({ providers: { [c.provider]: { baseUrl: `http://127.0.0.1:${server.address().port}`, api: "anthropic-messages", apiKey: c.key, models: [{ id: c.model }] } } }));
const child = spawn(command[0], command.slice(1), { stdio: "inherit", env: process.env });
const status = await new Promise((resolve) => {
child.on("error", () => resolve(1));
child.on("exit", (code, signal) => resolve(code ?? (signal ? 1 : 0)));
});
server.close();
process.exit(status);
}
const self = fileURLToPath(import.meta.url);
let invoked = null;
try { invoked = process.argv[1] ? realpathSync(process.argv[1]) : null; } catch {}
if (invoked === self) main(process.argv.slice(2)).catch(() => process.exit(1));
+14
View File
@@ -0,0 +1,14 @@
// #1873 pi-unpin S6: a second -e extension, for the fit check's model run only (pi-fit.mjs); no
// session loads it. pi loads it after the gate's, so for a read of THROW_NAME the gate has allowed
// the read before this handler throws. The run then shows that a throw in a tool_call handler
// stops the tool (S0 line 2: a gate that crashes blocks). It imports nothing, since it runs inside
// the fit check's sandbox.
export const THROW_NAME = "throw.txt";
export const THROW_TEXT = "fit check: the tool_call handler threw";
export default function fitThrow(host) {
host.on("tool_call", async (event) => {
if (event?.toolName === "read" && event.input?.path === THROW_NAME) throw new Error(THROW_TEXT);
return undefined;
});
}
File diff suppressed because it is too large. Load diff
+10 -5
View File
@@ -41,7 +41,7 @@ import { createServer, connect } from "node:net";
import { basename, join } from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { Client } from "../../bus/src/client.mjs";
import { LENGTH_FAILURE_NOTICE_PREFIX } from "./failure-notice.mjs";
import { LENGTH_FAILURE_NOTICE_PREFIX, failureBody, secretValues, turnCause } from "./failure-notice.mjs";
import { ID_PATTERN } from "../../organization/src/vocabulary.mjs";
import { firstOutputMarker, firstTurnMarker } from './first-output.mjs';
import { GUARD_CODES, FIXED_TEXTS } from "../../library/src/reply-protocol.mjs";
@@ -508,7 +508,12 @@ function runTurn({ session, request, n, env, onChild }) {
const missing = missingCommand(stderrTail(errPath));
return resolve({ ok: false, reason: `adapter exited 127${missing ? ` (command not found: ${missing})` : ""}` });
}
if (code !== 0) return resolve({ ok: false, reason: signal ? `adapter killed by ${signal}` : `adapter exited ${code}` });
if (signal) return resolve({ ok: false, reason: `adapter killed by ${signal}` });
if (code !== 0) {
// #1873 chat-fail F1: an adapter that ends on an error result prints it on stdout.
const cause = turnCause(out, secretValues(env));
return resolve({ ok: false, reason: `adapter exited ${code}`, ...(cause ? { cause } : {}) });
}
// A "marker" harness (row 65: the module's turnEnd()) writes the marker when its turn ends.
if (session.turnEnd === "marker" && !existsSync(marker)) return resolve({ ok: false, reason: `${session.harness} exited before the turn ended (no turn marker)` });
resolve({ ok: true, text: out.trim() });
@@ -801,11 +806,11 @@ export async function main(runDir, { stdin = process.stdin, env = process.env, l
catch (e) { log(`runner: failure record for ${m.id} failed: ${e.code ?? e.message}`); if (ended(e, false)) break loop; }
continue;
}
if (!turn.ok) log(`runner: turn ${n} for message ${m.id} failed: ${turn.reason}`);
if (!turn.ok) log(`runner: turn ${n} for message ${m.id} failed: ${turn.reason}${turn.cause ? ` (${turn.cause})` : ""}`);
try {
if (!isLengthFailureNotice(m) && m.class !== "RESULT" && (m.class === "REQUEST" || ["human", "tester", "user"].includes(m.from_role) || m.in_reply_to_class !== "RESULT")) {
if (!library) {
const body = turn.ok ? turn.text || "(the session gave no answer)" : `The turn failed: ${turn.reason}`;
const body = turn.ok ? turn.text || "(the session gave no answer)" : failureBody(turn);
await client.call("message.send", { to: m.from_role, body: clip(body) + unslop.note(), class: "RESULT", in_reply_to: m.id });
unslop.sent();
} else {
@@ -858,7 +863,7 @@ export async function main(runDir, { stdin = process.stdin, env = process.env, l
await client.call("message.read", { id: m.id });
continue;
}
if (!retry.ok) log(`runner: retry turn ${n} for message ${m.id} failed: ${retry.reason}`);
if (!retry.ok) log(`runner: retry turn ${n} for message ${m.id} failed: ${retry.reason}${retry.cause ? ` (${retry.cause})` : ""}`);
try { await send(retry); }
catch (again) {
if (!GUARD_CODES.includes(again.code) && again.code !== "request-too-large") throw again;
+2 -1
View File
@@ -4,7 +4,8 @@ import { createHash } from "node:crypto";
import { mkdirSync, readFileSync, statSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import { BundleError, buildBundle, sessionModel, SOURCES } from "../src/bundle.mjs";
import { HARNESS, REPO, REPO_SKILLS, resolvedFor, scratch } from "./helpers.mjs";
import { HARNESS, REPO, REPO_SKILLS, resolvedFor, scratch, harnessReady } from "./helpers.mjs";
await harnessReady(); // #1873 re-check 7: buildBundle reads a passed fit check
import { decide } from "../src/gate.mjs";
import { resolveInstance, validateBusinessDocument } from "../../organization/src/index.mjs";
import { businessDoc, REPO_ROLES, systemFor } from "../../organization/tests/helpers.mjs";
Loaded 100 of 263 files, more files were not shown because too many files have changed in this diff. Show more