jason.woltje
c216f47dbb
Adds the harness plug-in kind (interface 1). packages/plugins/harness/pi and claude-code hold what core branched on: the adapter, thinking levels, preflight, version probe, sandbox mounts, builtins line, typed tool names, gate timeout, bundle files, adapter env, turn end and session-files path. Core reaches a module only through the loader. A session's route is the instance's harness, model and thinking vars, or the three fallback. vars; a missing one refuses no-route, and the system file's execution keys are never read for a session. The launcher checks the primary route, then the fallback: harness-off, harness-refused, thinking-unsupported, then preflight. A failed preflight falls back when a fallback is set, and any refusal of the fallback names the primary's failure too. Account checks run only when the route's module counts an account. The manifest records route, routeReason and harnessVersion, replacing piVersion and claudeVersion. pi gets --thinking, Claude Code --effort, both from MOSAIC_THINKING. The business file refuses a partial fallback. routes and routes.set are human-only bus verbs outside ORGANIZATION_KEYS; checkRoutes is shared by routes.set, instances.set and limits.set and checks only changed routes. The chief of staff changes an instance's model, and the assistant's whole vars (Q169), through limits.set, so its route write runs the same checks. A model outside launch.models refuses unknown-model, primary or fallback, and routes.set passes that code through. A new run binds only on a switched-on harness module name and records its transcript path, which the usage reader uses. Settings gains a Routing table with six selects per instance and loses the Role models fieldset; it posts changed rows to /api/routes. webui/src/bus.mjs is unchanged. Until Jason's `mosaic plugins set` adds "harness": ["claude-code"], a new launch refuses harness-off.
309 lines
17 KiB
Markdown
309 lines
17 KiB
Markdown
# Plug-ins
|
|
|
|
The plug-in layer (PRD REQ-PLUG-1, build step 6b, Gitea #1626). Every part
|
|
the core calls out to is a plug-in kind. A module of a kind is a directory
|
|
in this package, and the organization file says which modules are switched
|
|
on. No core module imports a plug-in module by name; calls go through the
|
|
loader.
|
|
|
|
| Path | What it is |
|
|
|---|---|
|
|
| `src/kinds.mjs` | the kind registry and the check for the organization file's `plugins` key |
|
|
| `src/loader.mjs` | loads the switched-on modules from this tree |
|
|
| `<kind>/<name>/index.mjs` | one module of a kind |
|
|
|
|
## Kinds
|
|
|
|
| Kind | Interface | Switch | Exports |
|
|
|---|---|---|---|
|
|
| `usage-source` | 1 | list | `interfaceVersion`, `readTurn` |
|
|
| `limit-check` | 2 | list | `interfaceVersion`, `mayLaunch`, `mustStop`, `report` |
|
|
| `search-source` | 1 | one | `interfaceVersion`, `search` |
|
|
| `channel` | 1 | list | `interfaceVersion`, `dispatch` |
|
|
| `harness` | 1 | list | `interfaceVersion`, `adapter`, `levels`, `preflight`, `version`, `countsAccount`, `mounts`, `builtinWeb`, `builtins`, `typedName`, `reliesOn`, `gateTimeout`, `bundle`, `env`, `turnEnd`, `transcript` |
|
|
|
|
Every module exports `interfaceVersion`, equal to its kind's interface
|
|
version, plus the kind's functions. PRD REQ-LIMIT-4 says what the two kinds
|
|
do: a usage source reads what a session used after each turn, and a limit
|
|
check answers whether a launch may start (`mayLaunch`), whether a running
|
|
session must stop at the end of its turn (`mustStop`), and what each limit
|
|
has counted (`report`). Row 124 set the usage source's arguments and ships
|
|
the transcript source. Row 125 set the limit check's at interface 2 (lead
|
|
decision 150) and ships `token-limit`; a module built for interface 1 is
|
|
refused with `interface-version`.
|
|
|
|
A `search-source` answers the broker's `web.search` (REQ-HARN-2, REQ-PLUG-2, decision 0b81b0c5): `search(settings, query, deps, { signal })` resolves `{ query, total, results: [{ title, url, snippet }] }` or throws a `WebRefusal`; `signal` is an `AbortSignal` the broker fires at its deadline, and a module must stop its request when it does. The kind switches on one module, and the module's name is the source: `searxng-url` queries the operator's SearXNG at the business var `web.searxng`; `searxng-bundled` queries the SearXNG the Stack installs, at a fixed loopback address (the install is REQ-INST-1's first component); `harness-builtin` is served by the harness itself: a granted claude-code role gets WebSearch and WebFetch, the broker serves no web verbs, and `search` only refuses. The choice and every change to it are the `plugins` setting, logged in `changes.jsonl` by the plug-in setter.
|
|
|
|
Adding a kind, or changing a kind's exports, interface version or switch,
|
|
is a core change: a reviewed edit to `src/kinds.mjs`, with the code that
|
|
calls the kind.
|
|
|
|
## Adding a module
|
|
|
|
1. Pick a name matching `^[a-z][a-z0-9-]*$`.
|
|
2. Create `packages/plugins/<kind>/<name>/index.mjs`. A real directory and
|
|
a real file: the loader refuses a symlink anywhere from
|
|
`packages/plugins` down, `index.mjs` included.
|
|
3. Export `interfaceVersion` and every function the kind lists.
|
|
4. Land it like any other change. A running stack sees it only after a
|
|
re-pin, because `scripts/stack-copy.sh` copies the whole
|
|
`packages/plugins` directory into the pinned copy.
|
|
5. Switch it on in the organization file's `plugins` object, under its
|
|
kind: `mosaic plugins` shows the setting and its revision, and
|
|
`mosaic plugins set --file PATCH` (`{ "revision": ..., "plugins": {...} }`)
|
|
changes it, one kind at a time: the patch names the kinds it changes, a kind
|
|
with `null` is removed, and a kind it doesn't name keeps its setting. The
|
|
change goes through the broker's human-only `plugins` and `plugins.set`
|
|
verbs. The setter refuses a bad shape, a name with no module in this tree
|
|
(`unknown-plugin`), an unknown kind, an empty patch and a stale revision, and logs actor and old and new
|
|
names in `changes.jsonl`. A running broker reads the setting at bus start
|
|
(REQ-WEB-3's reload will change that), so restart the bus after a switch.
|
|
|
|
Adding a module changes no core file.
|
|
|
|
## The organization file's `plugins` key
|
|
|
|
One key per kind. A list kind takes a list of names; a kind that switches
|
|
on one module takes one name. An absent key switches the kind off.
|
|
|
|
```json
|
|
"plugins": { "usage-source": ["meter"], "limit-check": [] }
|
|
```
|
|
|
|
`validateBusinessDocument` checks the shape: known kinds, the name pattern,
|
|
no name twice. It doesn't check that a module exists; the loader does that
|
|
when it loads. The setter writes through the organization writer, which
|
|
logs actor and old and new names in `changes.jsonl` like a limit change. A
|
|
refused write throws `OrganizationError` (`invalid-plugins` or
|
|
`invalid-request`) and logs nothing.
|
|
|
|
## The loader
|
|
|
|
```js
|
|
import { createLoader } from "../packages/plugins/src/loader.mjs";
|
|
|
|
const loader = createLoader({ log: entry => record(entry) });
|
|
const { loaded, refused } = await loader.load(business.plugins);
|
|
loader.modules("limit-check"); // the switched-on modules, in order
|
|
loader.loaded(); // { kind: [names actually switched on] }
|
|
```
|
|
|
|
The loader finds this package from its own location and takes no path from
|
|
config. Until Jason rules on Vikunja #286 it loads modules from this tree
|
|
only (lead decision 143). For each name it checks, in order:
|
|
|
|
1. the name matches `^[a-z][a-z0-9-]*$` (`bad-name`);
|
|
2. `packages/plugins`, `<kind>`, `<name>` and `index.mjs` exist, are a
|
|
directory or a regular file as expected, and are not symlinks (`missing`,
|
|
`symlink`); ancestors of `packages/plugins` are not checked;
|
|
3. just before import, `realpath(index.mjs)` is still the expected path
|
|
(`symlink`);
|
|
4. the module imports (`load-failed`);
|
|
5. it has every export, and the kind's functions are functions
|
|
(`missing-export`);
|
|
6. `interfaceVersion` equals the kind's (`interface-version`).
|
|
|
|
At steps 2 and 3, a path that can't be read for another reason, such as a
|
|
permission error, refuses as `unreadable`.
|
|
|
|
A refusal refuses the whole kind for that load: the kind keeps the modules
|
|
it had, and the loader calls `log` with `{event: "plugin-refused", kind,
|
|
name, code, message, kept}`. A kind refused on its first load has nothing
|
|
switched on. An unknown kind refuses only itself; a `plugins` value that
|
|
isn't an object refuses every kind. Loads run one at a time.
|
|
|
|
Limits of the checks:
|
|
|
|
- A module runs with the host's authority once imported. The checks keep
|
|
the loader inside this tree; they don't sandbox the module.
|
|
- A module's top-level code runs at import, before the export and
|
|
interface-version checks. A module refused for `missing-export` or
|
|
`interface-version` has still run, and Node's cache keeps it.
|
|
- Someone who can write this tree can swap a file between the realpath
|
|
check and the import. The checks catch mistakes and stale links, not a
|
|
writer racing the host as the same user.
|
|
- Node caches a module by URL, so a changed `index.mjs` is not re-read
|
|
until the process restarts.
|
|
- The bus host wires the loader: row 124 for usage sources, row 125 for
|
|
limit checks, row 65 for harness levels. The launcher loads a launch's
|
|
harness itself.
|
|
|
|
## Harness (row 65)
|
|
|
|
A managed session's harness is a module under `harness/`: `pi` and
|
|
`claude-code` ship. Core reaches a harness only through the loader, so the
|
|
launcher, the bundle, the runner and the usage reader name no harness. The
|
|
host calls the module and writes its answers into `session.json` as data;
|
|
the sandboxed runner never imports it. `src/kinds.mjs` lists what each
|
|
export returns:
|
|
|
|
- `adapter()` is the absolute path of the adapter script the runner runs.
|
|
- `levels()` lists the thinking levels the harness applies: pi takes `off`,
|
|
`minimal`, `low`, `medium`, `high`, `xhigh` and `max` as `--thinking`,
|
|
Claude Code takes `low`, `medium`, `high`, `xhigh` and `max` as
|
|
`--effort`.
|
|
- `preflight({env})` returns `null` or `{code, message}`: pi checks its
|
|
binary in `node_modules` (`harness-binary-missing`), Claude Code its
|
|
session credential (`no-session-credential`) and `MOSAIC_CLAUDE_BIN`.
|
|
- `version({env})` is what the manifest records as `harnessVersion`.
|
|
- `countsAccount()` says whether its launches spend `launch.account` and get
|
|
the row 126 account checks.
|
|
- `mounts`, `builtinWeb`, `builtins`, `typedName`, `reliesOn`,
|
|
`gateTimeout`, `bundle`, `env`, `turnEnd` and `transcript` give its
|
|
sandbox binds, web, tool names, S0 lines, bundle files, adapter variables,
|
|
how a turn ends and where its transcripts land.
|
|
|
|
`plugins.harness` switches harnesses on, and an instance's `harness` and
|
|
`fallback.harness` vars each name one. A launch on a harness that isn't
|
|
switched on refuses `harness-off`, primary or fallback, and an off primary
|
|
never falls back. The launcher then loads the route's one module through a
|
|
fresh loader, so an earlier load's module is never reused. A module the
|
|
loader refuses fails the launch with `harness-refused`, naming the module
|
|
and the loader's code. The switch gates launches and writes, not loads: the
|
|
bus host loads every harness module in the tree to read its thinking levels,
|
|
on or off. The broker binds a new run only on a switched-on harness; a run
|
|
an earlier broker bound binds again.
|
|
|
|
Writes check it too. `routes.set`, `instances.set` and `limits.set` refuse a
|
|
changed route whose harness is off (`harness-off`), whose module didn't load
|
|
(`harness-refused`), or whose module doesn't apply its thinking level
|
|
(`thinking-unsupported`). A route the write doesn't change isn't checked,
|
|
so a model change still saves while `plugins.harness` is unset.
|
|
|
|
A new harness is a new module directory and a `plugins set` patch: no core
|
|
file changes. `tests/harness.test.mjs` launches a fixture module that way,
|
|
and greps every `packages/*/src` outside this package for a quoted harness
|
|
name. The exceptions are the tool gates (`harness/src/gate.mjs`,
|
|
`claude-gate.mjs`, `pi-extension.mjs`), the human-proof check
|
|
(`bus/src/human.mjs`), the engines that start pi themselves
|
|
(`conversation/src/`, `discord/src/`) and the process scan
|
|
(`control-board/src/scan.mjs`). The test fails if an exception stops naming
|
|
a harness, so the list stays true.
|
|
|
|
Switching harnesses on is a `mosaic plugins set` patch. The list replaces
|
|
the kind's whole setting, so it names every harness that stays on:
|
|
|
|
```json
|
|
{ "revision": "<from mosaic plugins>", "plugins": { "harness": ["pi", "claude-code"] } }
|
|
```
|
|
|
|
## Transcript usage source (row 124)
|
|
|
|
`usage-source/transcript` now ships interface 1's `readTurn({harness, root})`.
|
|
The bus brackets each actual harness turn, reads all sources through the loader,
|
|
and appends baselines and outcomes. See that module's README for JSONL formats,
|
|
security/resource bounds and the best-reading limitation. Enable transcript
|
|
through the human plug-in settings writer; a restart/re-pin remains a
|
|
separate authorized window.
|
|
|
|
## Limit checks (row 125)
|
|
|
|
Interface 2. Core calls each switched-on check through the loader only, in
|
|
the order the `plugins` setting lists them, with a copy of its arguments:
|
|
|
|
```
|
|
mayLaunch({business, limits, subject, records, now}) → {allow: true} | {allow: false, hit}
|
|
mustStop({business, limits, subject, records, now}) → {stop: false} | {stop: true, hit}
|
|
report({business, limits, records, now}) →
|
|
[{path, limit, since, count, kinds, unknownTurns} | {path, limit, since, notEnforced}]
|
|
```
|
|
|
|
- `limits` is the business's `launch.limits`. `subject` is
|
|
`{instance, provider, model, account}`. `records` are the per-turn usage
|
|
records the usage sources wrote (`usage.mjs` `turnRecords`), each
|
|
`known` with its four kinds or `unknown`. `now` is an ISO time.
|
|
- `hit` is `{path, limit, count, unknownTurns}`. Its path must name a limit
|
|
in `limits` with that limit. A count is a safe integer or a digit string.
|
|
- The launcher asks `mayLaunch` before every launch, through the bus
|
|
(`mayLaunch` launch op). A refusal is `limit-reached` with the hit as its
|
|
message, or `limit-check-failed`. The bus asks `mustStop` after each
|
|
turn's usage end. A stop lets the turn's result and reads land, then
|
|
revokes the role claim (or releases the thread claim) with by
|
|
`limit-check` and a reason `limit: <check> <path> limit <n> count <c>`;
|
|
`session.ended` and the launcher's end record carry that reason.
|
|
- `report` feeds Settings and `mosaic limits`. Core composes one entry per
|
|
limit: `counted`, `not-enforced` (no check reported a count; a check's
|
|
`notEnforced` text is the reason) or `check-failed`.
|
|
|
|
Fail closed (lead decision 150): a check that throws, answers in the wrong
|
|
shape, answers with a promise, or is switched on but not loaded refuses the
|
|
launch, stops the session at the end of its turn, and shows every limit as
|
|
`check failed: <code>`. The code is the check's own thrown `code` when it
|
|
matches the plug-in code shape, else `check-threw` or `malformed-answer`.
|
|
Both refusals and stops are logged with the check and the code or hit.
|
|
|
|
### token-limit
|
|
|
|
`limit-check/token-limit` enforces `launch.limits.tokens`:
|
|
|
|
```json
|
|
"tokens": {
|
|
"instances": {"coder": {"limit": 5000000, "since": "…"}},
|
|
"providers": {"anthropic": {"limit": 50000000, "since": "…"}},
|
|
"models": {"anthropic": {"claude-opus-5-5": {"limit": 20000000, "since": "…"}}},
|
|
"keys": {"team-key": {"limit": 10000000, "since": "…"}}
|
|
}
|
|
```
|
|
|
|
A count is the sum of all four kinds over the subject's known records since
|
|
`since`; unknown turns add nothing and are counted apart. The limits writer
|
|
sets `since` when a limit is added; a changed limit keeps it, a caller's
|
|
`since` that differs is refused, and only a `reset` update (a list of paths,
|
|
`mosaic limits reset tokens providers anthropic`, Settings, or the chief's
|
|
`limits.set`) moves it to now. There is no ceiling (Q150). A key matches a
|
|
record's account; while no record names one, a key limit reports as not
|
|
enforced. See `limit-check/token-limit/README.md`.
|
|
|
|
### daily-quota
|
|
|
|
`limit-check/daily-quota` (row 126) enforces `launch.limits.tokens.daily`,
|
|
one quota per account, each with the `reset` time of day and the IANA
|
|
`timeZone` its day starts in:
|
|
|
|
```json
|
|
"daily": {"jason-max": {"limit": 40000000, "since": "…", "reset": "05:00", "timeZone": "America/Chicago"}}
|
|
```
|
|
|
|
A count is the sum of all four kinds over the account's known records since
|
|
the later of the day's start and `since`. Only a Claude Code launch carries
|
|
an account; a pi launch's is null, so no quota covers it. The count is this
|
|
stack's own use of the account, not its use anywhere else, and `mosaic
|
|
limits` and Settings say so. See `limit-check/daily-quota/README.md`.
|
|
|
|
## Human project/thread channels
|
|
|
|
`channel` modules carry project/thread operations between a human surface and
|
|
its existing core bus caller. The shipped `cli` and `webui` modules implement
|
|
`dispatch({ verb, args }, { call })`, returning the bus result or refusal
|
|
unchanged. Only the fixed project/thread operations are supported; CLI also
|
|
uses message.receive/read for thread talk/follow. Modules do not create a human
|
|
proof, choose a business/owner, spawn the shim or write organization settings.
|
|
The broker still sees the proved transport as `via: 'cli'` for both surfaces.
|
|
Discord is not connected and no Discord module ships here.
|
|
|
|
The setting is `"plugins": { "channel": ["cli", "webui"] }`. An absent key or
|
|
an empty list disables both converted chat surfaces. No silent default enables
|
|
them. CLI off refuses with exit 3; web chat off returns 503, both naming recovery.
|
|
Other CLI administration and Console views are independent. CLI project
|
|
init/check and non-thread talk/inbox are core operations, outside this conversion.
|
|
|
|
Each chat operation reads the existing plugins setting, loads only its channel
|
|
selection through the in-tree loader, captures a frozen `snapshot('channel')`
|
|
array of `{ name, module }`, then dispatches without a lock. A slow send does
|
|
not block a list. A captured array never changes after a reload; loaded() and
|
|
modules() keep their existing interfaces. Invalid module settings retain the
|
|
loader's last good selection; the first refused load has nothing enabled.
|
|
|
|
The recovery commands never need a channel: from a human shell, run
|
|
`mosaic plugins --business BUSINESS --json`, make a patch with its revision and
|
|
`{"channel":["cli","webui"]}` as the plugins value, then run
|
|
`mosaic plugins set --business BUSINESS --file PATCH`. This uses the existing
|
|
proved human transport and organization writer, with revision checks and actor
|
|
receipts. The next chat operation reads the restored selection. Disabling a
|
|
channel cannot disable this write path. It is not a repair bypass for an invalid
|
|
organization file or unavailable bus.
|
|
|
|
For the re-pin introducing this kind, Jason enables both channels through
|
|
plugins.set immediately after activate in Mos's window, then smokes CLI thread
|
|
listing and Console chat. No development seat performs that live step.
|