Files
stack/packages/discord/README.md
T
jason.woltjeandClaude Opus 5.5 6c06a6f358 fix(discord): row 25 SetSpark key reaches pi and the connector (#1509)
resolveToolRoots dropped tools.setspark, so the record verbs never reached
pi's --tools list and the connector never built its SetSpark client. The
resolved config now carries only baseUrl, keyFile, principal and timeoutMs,
the keys the extension's loadSetsparkConfig accepts; the extension restores
the response cap. The connector's client uses the validated binding object,
which keeps the cap. README: principal is required, timeoutMs is optional.

Test (failing first): binding -> resolveToolRoots -> JSON -> loadToolsConfig
gives the binding's config, and the verbs reach enabledToolNames. Eight
suites green on an index export. Rocko approved
(agents/rocko/work/row25-setspark-fix-review-2026-09-26.md, a28df89e).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-26 15:57:55 -05:00

26 KiB

discord

The Discord connector: one seat's conversation reachable from listed channels of one Discord server, chat plus file tools confined to declared folders: reads everywhere, writes only where a root allows them. Issue #1509, briefs docs/plans/2026-09-13_discord-connector-pilot.md, docs/plans/2026-09-14_discord-readonly-tools.md and docs/plans/2026-09-16_discord-write-and-web-tools.md. Plain ESM, no dependencies, Node 24 or newer, built-in WebSocket and fetch.

A Discord channel is one more interface onto a seat's conversation, the same class of thing as the terminal and the WebUI session chat. The connector owns a pi --mode rpc engine today; that module is the one to swap when the CHAT-03 conversation controller exists.

Commands

scripts/discord.sh check <binding>
scripts/discord.sh run   <binding> [--supervised]
scripts/discord.sh stop  <binding>
scripts/discord.sh unlock <binding>
scripts/discord.sh recover <binding>
scripts/discord.sh reload <binding>
scripts/discord-service.sh render | install | uninstall | status <binding>

<binding> names <dataRoot>/discord/<binding>.json. The wrapper passes --repo for you; the CLI also takes --config PATH.

  • check validates the binding, the token file (0600, regular, not a symlink), the context files and the pi install; reads the bot, the guild and every listed channel over REST; opens one gateway connection, waits for READY, closes it. Nothing is sent to a channel. Close code 4014 is reported as the message-content intent not being granted in the developer portal.
  • run refuses when STOP exists or an unresolved delivery cannot be reconciled. Otherwise it starts pi, connects, and serves turns until SIGTERM, SIGINT or stop. Run it under the service unit below, or by hand in a tmux window.
  • stop writes STOP and sends SIGTERM to the process in run.lock, only when that pid is alive and both its start time and the boot id match the recorded ones; a reused pid, a pid from a previous boot, or a pid whose identity cannot be read right now is never signaled. The current turn finishes or times out, then the process exits. Remove STOP to run again.
  • unlock writes STOP, then removes a run.lock whose owner is gone (crash, reboot, a start interrupted before it published its record). It refuses while the owner is running; use stop for that. It also refuses, removing nothing, when the pid is alive and its identity cannot be established: the record predates the boot id (an upgrade over a running connector), a recorded value is not a start tick or a boot id, or /proc cannot be read right now. Once that pid is dead, unlock clears it. A record file that exists but cannot be parsed is never removed or claimed over; inspect it by hand. STOP is the gate that serializes cleanup with starts: a run re-checks STOP after publishing its record and releases itself if it is there, so nothing that starts during an unlock can hold the binding. run never reclaims a stale lock on its own; it refuses and names this command. Remove STOP to run again.
  • recover is the supervised pre-start; run --supervised performs it first, in the same process, and that is the form the service unit uses, because systemd honours a never-retry exit status only from the main process. It refuses, with exit 3 and touching nothing, while STOP is present or the binding is held by a live process, an alive pid whose identity cannot be verified, or an unreadable record. A lock whose owner is gone, or that has no record, it clears the way unlock does, then it removes the STOP it wrote for that so the run that follows can claim. It removes only a STOP that consists of the one line it wrote itself; a brake an operator wrote at any point, even during the recovery, stays and the start is refused. Nothing automatic ever removes an operator's STOP.
  • reload applies an edited binding to the running connector without a restart. It validates the file first (an invalid file exits 2 and nothing is signaled), then sends SIGHUP to the live owner in run.lock (no live owner exits 1; the file applies at the next start). The process re-reads the file and swaps guildName, channels, users and limits in place; a channel that is new to the binding is read over REST and must be in the bound guild. name, seat, guildId, botUserId, tokenFile, engine and context are fixed for the life of the process, because the engine and its prompt are launched once and the token is read once; a change there, an invalid file or a failed channel lookup refuses the reload and keeps the old binding. The turn in flight finishes under the limits it started with; the next admission uses the new binding. Every attempt is one line in reloads.jsonl, applied with the differences by id or refused with the reason, and one line in the log. Nothing is sent to Discord. Under the service unit, systemctl --user reload mosaic-discord@<binding> sends the same signal.

Exit codes: 0 ok, 1 operation failed, 2 invalid data or configuration, 3 refused by a brake (STOP present or the binding held; a supervisor must not retry), 4 usage.

Service unit

scripts/discord-service.sh install renders packages/discord/systemd/[email protected] with the repository path and the directory of node, writes it to ~/.config/systemd/user/[email protected] (temp file, then rename; --dir DIR for another place, --no-reload to skip daemon-reload) and prints the commands that follow. One instance per binding:

systemctl --user enable --now mosaic-discord@<binding>   start now and at login
systemctl --user status mosaic-discord@<binding>
journalctl --user -u mosaic-discord@<binding> -f          the log (stderr of `run`)
systemctl --user stop mosaic-discord@<binding>            SIGTERM; the turn in flight finishes; restartable
scripts/discord.sh stop <binding>                         the brake: writes STOP; the unit stays down until STOP is removed
scripts/discord-service.sh status <binding>               unit state, STOP, run.lock

What the unit does: ExecStart runs run --supervised, Restart=on-failure with 15 seconds between tries and at most five in ten minutes, and RestartPreventExitStatus=3 so a brake is never retried. A crash (any other non-zero exit, a signal, an engine that died) restarts, and the supervised run clears the dead lock on its way in. systemctl --user stop sends SIGTERM only; the connector exits 0 and no STOP is written, so the next start needs no hand. The stop timeout is 3700 seconds, the largest turn timeout a binding may set plus margin; a normal stop takes as long as the turn in flight. The unit never reads the binding or the token; run does, at runtime, as before. Surviving logout and reboot needs loginctl enable-linger. render prints the unit without writing it; uninstall refuses while an instance is active. The log goes to journald: run writes no message text to stderr, only ids, counts and state.

The binding

Deployment policy for one seat on one server. It carries Discord ids of real people, so it lives under the data root at mode 0600 and is never committed. fixtures/binding.example.json is the shape with placeholder ids; the schema is src/binding.mjs.

Field Meaning
bindingVersion 1
name, seat binding name (matches the file name) and the seat it serves
guildId, guildName, botUserId the one server and the bot identity check confirms
tokenFile absolute path to the bot token, 0600, read into memory at start, never printed or journaled
channels[] {id, name, mode}; open answers every message, mention only when the bot is mentioned; threads inherit the parent's mode
users[] {id, name, channels?}; the only authors that get a turn. channels is an optional allowlist of listed channel ids; absent means every listed channel, present means those and their threads only, everything else is dropped as channel-not-for-user
engine provider, model, thinking for pi
limits turnsPerDay (200), turnTimeoutSeconds (180), replyChunkChars (1900), inboundMaxChars (4000)
context.files[] files appended to pi's system prompt in order, repository-relative and inside the repository (no absolute paths, .. or symlinks); the Discord block is added after them
tools optional. roots[] of {name, path, write?, git?}: absolute directories the seat may read through list_dir, read_file and search; a root with "write": true may also be written through write_file and edit_file; a writable root that is a git work tree may carry git {branch, identity, tokenFile, author, protocol?} and gains git_status, git_commit, git_pull and git_push (protocol: "vault" adds reserve_id); maxFileBytes (262144), maxCallsPerTurn (8); web (optional) {searxng, maxFetchBytes} enables web_fetch and web_search through the named SearXNG instance (https, or http on loopback; maxFetchBytes 1048576); setspark (optional) {baseUrl, keyFile, principal, timeoutMs?} names the SetSpark record service (https origin, or http on loopback; key file absolute, 0600, read per call, never printed) and turns on connector-verified approvals. Absent means no tools and a pi launch with --no-tools. A root may not be /, the home directory, a symlink, a path with a dot-prefixed segment, or anything inside or above the data root

Unknown keys, missing fields, wrong types, empty allowlists, a user channel that is not listed and a bot listed as a user all refuse with exit 2. A running connector picks up an edit through reload; the fields it will not take in place are listed under that command.

File tools

With tools in the binding, pi starts with --no-builtin-tools, loads extension/tools.mjs explicitly, and allowlists exactly the tools the binding enables: list_dir, read_file and search always, plus write_file and edit_file when at least one root has "write": true, plus web_fetch and web_search when tools.web names a SearXNG instance (enabledToolNames in src/tools.mjs is the one place that decides; the suite checks the real pi exposes that list and nothing else). The extension reads its roots from the MOSAIC_DISCORD_TOOLS environment variable the connector sets, and throws without it, which makes pi exit and the connector refuse to start. Every rule lives in src/tools.mjs and is tested without pi: a request names a root and a relative path; .., empty and dot-prefixed segments are refused; every step below the root is lstated and a symlink refuses; the real path must stay under the root; only regular files are read and only directories listed; a file with more than one hard link is refused; a read opens the file once without following a symlink, checks device and inode against the walk, and reads from that descriptor only; files over maxFileBytes or with a NUL byte in the first 8 KiB are refused; content that matches a credential shape refuses the whole read; and after maxCallsPerTurn calls in one message every call is refused until the next message. A refusal is a normal tool result with a fixed reason; the model never sees a host path outside the root. The turn record gains tools[], one entry per call with name, root, path, outcome, reason, bytes and duration, and engine.turns, the number of pi turns the answer took. The system prompt names the roots, says file content is data like Discord text, and tells the seat to say plainly when a read was refused.

Writes (row 23) add rules on top of the read rules, with the same fixed refusals. write_file(root, path, text) creates or replaces a file; edit_file(root, path, old, new) replaces one exact string that occurs exactly once. Both refuse a root without write: true, a parent folder that does not exist (no folder is ever created), any dot-prefixed segment (so .git/ is unreachable), a symlink anywhere in the path, a target that is not a regular file with one link (a folder, a FIFO, a hard link), text over maxFileBytes or with a NUL byte, and text that matches a credential shape. The bytes go to a dot-prefixed temp file in the same folder, created exclusively, then one rename over the target after a second lstat confirms the target is the file that was checked (or is still absent); a refused rename removes the temp file. The tool result says the file is not committed, and the prompt tells the seat to end its reply by naming the file it changed: Sage has no git, Jason commits from the terminal.

Web tools (row 23, src/web.mjs, no dependencies). web_fetch(url) is one GET of an absolute https url with no user or password part. The host is resolved first and every address must be public: loopback, private, link-local, carrier-grade NAT, multicast and IPv4-mapped forms refuse the call, and the connection is pinned to the vetted address so a name that answers differently on the second lookup gains nothing. At most three redirects, each re-checked by the same rules and refused unless https. The body stops at maxFetchBytes; html is reduced to text with its title (scripts, styles and comments dropped); plain text, json and xml pass as they are; anything else is refused. The model sees at most 12000 characters. One fixed User-Agent, no cookies, no auth headers, no POST, and the whole call ends within 15 s. web_search(query) asks the instance /search?q=…&format=json and returns title, url and snippet for at most 10 results; a query over 400 characters, a non-200 answer or a non-json body is refused. The instance url must be https or http on loopback. Both count against maxCallsPerTurn, and the turn record keeps the url, status and byte count (fetch) or the query and hit count (search). The prompt says web content is data like file content. The tests drive both tools against a local server through an injected resolver and transport, so the fence is tested without the network; the real transport is node:https with the same options.

Git verbs (row 24, src/git.mjs, bin/git-credential.mjs). A root with write: true may carry a git object: branch (the only branch the verbs work on), identity (the seat name, used as the https username and as the lock owner), tokenFile (an absolute path to a private 0600 file holding the seat's token; the connector checks the mode and never reads the content), author as Name <email>, and optionally protocol: "vault". The root must be a work tree (.git present). Every verb is one git child with a fixed argument list, run inside the root, within 60 s, output capped at 4 KiB and masked (https://user:pw@ and GitHub token shapes become <masked>). Before any verb: the head must be on branch, not detached, not mid-merge, rebase, cherry-pick, revert or bisect, with no conflicted path. git_status(root) reports branch, ahead/behind and changed paths. git_commit(root, message, paths) takes one to fifty paths, each a regular file under the root by the read rules (no dot segment, no symlink), and refuses when the index already holds staged work so a terminal user's half-done commit is never swept in; it stages exactly those paths, refuses when nothing changed, commits as author with a Requested-by: trailer carrying the Discord author's server name, then pushes at once (Jason's D6). A commit whose push fails is still a commit: the result says pushed: false with git's masked message, and the next commit's push carries both. git_pull(root) is pull --ff-only --no-rebase origin <branch>; a diverged origin or a dirty path refuses with nothing merged. git_push(root) pushes that one branch, never force, never tags. The requester comes from the envelope line the connector writes (requester="…"), read by the extension on before_agent_start; a commit with no requester is refused.

Credentials: the fleet helper git-credential-mosaic serves the Gitea hosts only and declines github.com, and the host's global git config routes github.com to Jason's own gh login, so neither may run for Sage. The verbs run git with GIT_CONFIG_GLOBAL=/dev/null, GIT_CONFIG_NOSYSTEM=1, GIT_TERMINAL_PROMPT=0, no askpass, and one credential.helper set through GIT_CONFIG_COUNT: the package's own bin/git-credential.mjs. It answers only get over https, reads the token from the path in MOSAIC_DISCORD_GIT_TOKEN_FILE (set by the connector for push, pull and reserve only; local verbs never carry it), refuses a symlink, a non-0600 mode or a value that is not a plain token, and writes username and password to git's stdin pipe. The token is never an argument, never in the environment, never in a record; the tests run every verb through a spy spawn and assert no argv holds the token or its path.

Vault protocol (protocol: "vault", the shared-signals record rules): write_file and edit_file take the clone lock for the path (tools/vault_lock.py lock, TTL 300 s) around the write and release it after; a lock held by another owner refuses the write with that owner's name. git_commit runs vault_lock.py check on the named paths and then tools/validate_vault.py, and refuses with the tool's first lines when either fails. reserve_id(root, prefix, title) runs vault_lock.py reserve (prefix BUS, PRJ, SS, DEC or REF; title up to 200 characters) and returns the id; the registry line goes into the next commit with the record. These scripts belong to the shared-signals repository, are run as fixed argv inside the root with the seat as owner, and never through a shell. The tests stand in small Python scripts with the same command line.

SetSpark record client (row 25, src/setspark.mjs and src/approvals.mjs, no dependencies). tools.setspark is validated at load like a git key. One HTTP core, callApi, sends JSON with Authorization: Bearer from the key file (one bare key line, or the mint's JSON output with its key field, stored as minted; re-read and re-checked on every call, so a rotation needs no restart), a fixed User-Agent, a 15 s timeout and a 256 KiB response cap, no redirects. A write carries an idempotency-key header, <principal>:<turn id>:<call index> for the model's verbs and <principal>:<event id>:<step> for the connector's own calls. Error bodies become refusals rendered from the status, the code and, on 409, current_revision and changed_fields; free text from the server is cut at 400 characters.

The model's verbs (contract: shared-signals stack/api/openapi.json at a5425a2), each one fixed path, registered by the extension only with a setspark key: record_list, record_get, record_create, record_update (carries the revision from record_get; a stale one is refused with the current revision and the changed fields), resolve_id, open_approval_request, get_approval_request, create_document (Outline). get_counters is left out. Arguments are checked before any request (record type from the fixed five, ids AA-1 shaped, property names lower snake case, a record under 32 KiB, a document under 20000 characters); a write outside a turn is refused, since no key can be formed. Writes carry a context object (turn id, the asserted requester's id and name, the client version) that the service records next to the verified key and never uses for authorization. A record renders as key: value lines cut at 6000 characters; a list shows at most 50.

Approvals. The model never asserts an approval. When a turn's tool calls include a successful open_approval_request, the connector posts the proposal as its own message (decision id, version, digest, who may approve, how) with an Approve button, appends opened to approvals.jsonl, and binds the message id to the request at the service. An approval is that button (INTERACTION_CREATE, answered with a deferred update, then the message is edited to "Approved by ." and the button is disabled once every approver has approved) or a reply to that message whose content is exactly approve. The author must be one of the request's approvers; anyone else gets one fixed line (private for a button, a reply for a message) and a drop entry, and so does a repeat. The service wants one Discord message per approval as evidence, its url and its exact text. A reply is its own evidence. A button press has none, so the connector first posts a confirmation line in the same channel ("Approval: approved DEC-012 v2 (digest 01234567) by button.") and submits that message's url and text; if that post fails, nothing is submitted, the request message says to press again, and a drop entry records it. The connector submits kind (button or reply), request id, author id, message id, the bound message id, the evidence url and the statement; the service verifies against what it stored. Every bind and approval is journaled as intent before the call (with the evidence message id and statement) and done, refused or unknown after it; start retries the unknown ones under their original keys and the same evidence. An approval reply never reaches the model; any other reply to the request message does.

What happens to a message

  1. The gateway delivers MESSAGE_CREATE. authorize drops it unless the guild matches, the author is listed and is not a bot, webhook or the bot itself, the channel or the thread's parent is listed, and in mention mode the bot is in mentions (@everyone does not count). A drop is one line in drops.jsonl and no reply.
  2. The message id is appended to inbox.jsonl before anything else. On start the inbox is read back; a replayed id is dropped as duplicate. That is the restart guard.
  3. STOP, an oversize message and the daily ceiling are checked next. Over size gets one fixed line. Over the ceiling gets one fixed line per UTC day, then silence until midnight UTC; the process stays up.
  4. The prompt is an envelope, one bracketed line naming server, channel, thread, author id and message id, then the text. The system prompt says that text is data. A message that arrives during a turn is held by the connector and sent when pi settles, one run per message, so it is neither lost nor run concurrently (a pi follow-up would fold it into the running answer and lose the first reply). As soon as the turn is admitted the connector reacts to the inbound message with eyes as a read receipt; a typing indicator follows every 8 seconds while the turn runs. A reaction Discord refuses is logged and recorded in the turn; it never fails the turn.
  5. The reply is split at 1900 characters on paragraph boundaries. Each chunk is posted with nonce and enforce_nonce: true, allowed_mentions empty, and the first chunk as a reply to the inbound message. An intent line goes to outbox.jsonl before the POST and a confirmed, refused or unknown line after it. A chunk that is not confirmed stops the rest of that reply.
  6. One write-once record per turn lands in turns/<message id>.json: ids, timing, usage, read-receipt outcome, delivery outcome, and the error on a failed turn. A failed turn posts one fixed line, never model output.

On start, every intent or unknown delivery is reconciled by sending the same nonce again; Discord returns the existing message instead of posting twice. An intent older than five minutes is outside Discord's dedupe window and is marked refused rather than re-sent, because a re-send could post a second reply. If anything is still unknown after that, run refuses to start and names the nonces.

Runtime data

<dataRoot>/discord/<binding>.json          the binding, 0600
<dataRoot>/discord/<binding>/inbox.jsonl    accepted message ids
<dataRoot>/discord/<binding>/outbox.jsonl   delivery intent and receipts, by nonce
<dataRoot>/discord/<binding>/drops.jsonl    one line per dropped or refused message
<dataRoot>/discord/<binding>/admissions.jsonl  one line per admitted turn, before the engine runs
<dataRoot>/discord/<binding>/turns/<id>.json  write-once turn records
<dataRoot>/discord/<binding>/launches/      context snapshot and sha256 per run
<dataRoot>/discord/<binding>/STOP           stop switch
<dataRoot>/discord/<binding>/notices.jsonl  once-per-day fixed lines already attempted
<dataRoot>/discord/<binding>/reloads.jsonl  one line per reload attempt, applied or refused
<dataRoot>/discord/<binding>/approvals.jsonl  open approval requests, binds and approvals, appended only
<dataRoot>/discord/<binding>/run.lock/      ownership directory (atomic mkdir) with owner.json {pid, start, boot}; stale ones need `unlock`
<dataRoot>/sessions/discord-<binding>/      the pi session, continued across runs

Directories are 0700, files 0600. Logs are append-only; turn records are written with O_EXCL and never rewritten.

Tests

scripts/test-discord.sh or node --test packages/discord/tests/. All offline: fake WebSocket and timers for the gateway, fake fetch for REST, a scripted stand-in for pi over stdio, a disposable data root. Groups: binding, authorization table, gateway (hello, identify, heartbeat, missed ack, op 7, op 9, close 4014), delivery and reconcile, engine (held prompt, timeout, malformed line, tool runs), restart replay, stop and ceiling, tools confinement, SetSpark config, key read and HTTP core against a local server playing the service, approvals (ledger, reply and button resolution, the connector flow with a fake api). The suite also starts the real pi offline three times, with no model call, to show the extension exposes exactly the three tools, the pilot flags expose none, and a missing MOSAIC_DISCORD_TOOLS makes pi exit.

Not in this piece

Repository writes, per-user tool gating, announcements, attachments, slash commands, DMs, per-thread sessions, more than one server or seat. Section 8 of the pilot brief keeps the list; the control-board row is darkwing's.