From 436ba6ed6be6cc2ec00a2f985c1ad2a6086705b5 Mon Sep 17 00:00:00 2001 From: Jason Woltje Date: Sun, 13 Sep 2026 14:39:11 -0500 Subject: [PATCH] feat(discord): systemd user service with a supervised run; brakes exit 3 and are never retried (#1509) QUEUE row 17, MVP iteration 2. scripts/discord-service.sh renders and installs mosaic-discord@ from packages/discord/systemd/. The unit's main process is `run --supervised`, which applies the new recover policy first: a lock whose owner is gone is cleared and only the STOP written for that is removed; an operator STOP or a held binding refuses with exit 3, which RestartPreventExitStatus never retries. `recover` is also a CLI verb. First cut used ExecStartPre and looped live, since systemd honours the never-retry status only from the main process; replaced and re-verified before any message traffic. Suite 40/40, 95 node tests. Co-Authored-By: Claude Fable 5.1 --- BUILD-LOG.md | 27 +++ docs/SESSIONS.md | 1 + docs/TOOLS.md | 17 +- .../2026-09-13_discord-connector-pilot.md | 14 ++ docs/plans/CURRENT.md | 1 + docs/plans/QUEUE.md | 4 + packages/discord/README.md | 59 +++++- packages/discord/src/cli.mjs | 44 +++- packages/discord/src/errors.mjs | 3 +- packages/discord/src/journal.mjs | 82 +++++++- .../systemd/mosaic-discord@.service.in | 35 ++++ packages/discord/tests/binding.test.mjs | 2 +- packages/discord/tests/recover.test.mjs | 192 ++++++++++++++++++ scripts/discord-service.sh | 139 +++++++++++++ scripts/test-discord.sh | 40 +++- 15 files changed, 630 insertions(+), 30 deletions(-) create mode 100644 packages/discord/systemd/mosaic-discord@.service.in create mode 100644 packages/discord/tests/recover.test.mjs create mode 100755 scripts/discord-service.sh diff --git a/BUILD-LOG.md b/BUILD-LOG.md index 1fec7774..8bb3abc7 100644 --- a/BUILD-LOG.md +++ b/BUILD-LOG.md @@ -2383,3 +2383,30 @@ node tests. Live check, 19:21 UTC: Jason sent a message in #sage-admin, the eyes reaction appeared on it before the reply, the turn record carries `receipt.ok` true, and a 0600 receipt sits in the private evidence directory. Row 15 done. + +## 2026-09-13 — Discord service unit (#1509, QUEUE row 17) + +Before: the Sage connector ran in a tmux window; a crash or a reboot left a +stale `run.lock`, and every rerun after `stop` needed STOP removed by hand. +After: `scripts/discord-service.sh install` renders and writes a systemd +user unit, `mosaic-discord@`, whose main process is +`discord.sh run --supervised`. The supervised run first applies +the new `recover` policy (also a CLI verb): a lock whose owner is gone is +cleared the way `unlock` does and only the STOP written for that is +removed; an operator STOP, or a binding held by a live or unverifiable +process, refuses with the new exit code 3, which the unit never retries. +`systemctl --user stop` is the soft stop (SIGTERM, no STOP); `discord.sh +stop` stays the brake. Tests: recover policy (clean, dead lock, no record, +operator STOP untouched, own leftover STOP, foreign line, brake written +during the unlock wins, live and unknown and invalid owners refuse without +writing STOP), CLI exit codes, unit render and install and uninstall, +`systemd-analyze verify`. Suite 40/40, 95 node tests. Loosening the STOP +ownership rule fails three tests. + +Correction, same day: the first cut used `ExecStartPre=discord.sh recover`. +Live, a refused pre-start went into systemd's auto-restart, because +`RestartPreventExitStatus` applies to the main process only. Caught on the +first start behind a brake, before any message traffic; replaced by the +supervised run as the main process and re-verified: SIGKILL recovered in +16 s with the dead lock cleared, brake held with no restart, released and +READY. The Sage seat now runs under the unit; the tmux session is gone. diff --git a/docs/SESSIONS.md b/docs/SESSIONS.md index 096317d4..ef6edbf5 100644 --- a/docs/SESSIONS.md +++ b/docs/SESSIONS.md @@ -251,3 +251,4 @@ are never rewritten or removed; corrections are new entries. - 2026-09-13 UTC — coordinator (Claude) — Discord connector pilot closed (#1509, QUEUE row 14): Jason ruled Gate H passed, replies read as Sage. DISCORD-USER.md wording corrected (unlisted senders are dropped silently, not refused with a reply). Records committed and pushed on Jason's authorization; connector left running for MVP iteration. - 2026-09-13 UTC — coordinator (Claude) — Discord read receipt (#1509, QUEUE row 15): eyes reaction on every admitted message at admission, best effort, outcome in the turn record; REST `react` plus connector and REST tests. `scripts/test-discord.sh` 28/28 (90 node tests). Committed; connector restarted to pick it up. Push on Jason's say-so. - 2026-09-13 UTC — coordinator (Claude) — Discord read receipt live check (#1509, QUEUE row 15): reaction confirmed on a live message at 19:21 UTC, turn record receipt ok true, private evidence receipt written, row 15 done. Committed; push on Jason's say-so. +- 2026-09-13 UTC — coordinator (Claude) — Discord service unit (#1509, QUEUE row 17): `run --supervised` and `recover`, exit 3 for brakes, `scripts/discord-service.sh`, systemd user unit; suite 40/40 (95 node tests); Sage seat migrated from tmux to `mosaic-discord@shared-signals`, kill and brake checks passed live. Committed; push on Jason's say-so. diff --git a/docs/TOOLS.md b/docs/TOOLS.md index f63c4682..11d93b18 100644 --- a/docs/TOOLS.md +++ b/docs/TOOLS.md @@ -151,9 +151,11 @@ usage. Details and the record's fields: `packages/seat/README.md`. ```bash scripts/discord.sh check -scripts/discord.sh run +scripts/discord.sh run [--supervised] scripts/discord.sh stop scripts/discord.sh unlock +scripts/discord.sh recover +scripts/discord-service.sh render | install [--dir DIR] [--no-reload] | uninstall [--dir DIR] [--no-reload] | status ``` One seat's conversation reachable from listed channels of one Discord @@ -167,12 +169,19 @@ reconciled. `stop` writes `STOP` and signals the running process. `unlock` writes `STOP` and removes a `run.lock` left by a dead process; it refuses while the owner is alive with an unverifiable identity or its record cannot be parsed. `run` refuses a stale lock rather than reclaiming it, and releases a -claim that meets `STOP`. Records +claim that meets `STOP`. `recover` is the supervised pre-start, and +`run --supervised` does it first in the same process: it refuses (exit 3) +while `STOP` is present or the binding is held, clears a lock whose owner +is gone and removes only the `STOP` it wrote for that. +`scripts/discord-service.sh install` renders and writes the systemd user +unit `mosaic-discord@.service` (one instance per binding, restart on +failure, exit 3 never retried, SIGTERM on `systemctl --user stop`). Records under `/discord//`: `inbox.jsonl`, `outbox.jsonl`, `drops.jsonl`, `admissions.jsonl`, `notices.jsonl`, write-once `turns/.json`. Suite: `scripts/test-discord.sh`. -Exit codes: 0 ok, 1 operation failed, 2 invalid data or configuration, 4 -usage. Details: `packages/discord/README.md`. +Exit codes: 0 ok, 1 operation failed, 2 invalid data or configuration, 3 +refused by a brake (a supervisor must not retry), 4 usage. Details: +`packages/discord/README.md`. ## Tools (host-side) diff --git a/docs/plans/2026-09-13_discord-connector-pilot.md b/docs/plans/2026-09-13_discord-connector-pilot.md index 203c4a21..7c8ef737 100644 --- a/docs/plans/2026-09-13_discord-connector-pilot.md +++ b/docs/plans/2026-09-13_discord-connector-pilot.md @@ -421,3 +421,17 @@ Recorded here because they refine a ruling or fill a gap the rulings left. is logged and recorded in the turn record as `receipt.ok: false` and never fails the turn. Needs the Add Reactions permission on the bot role. - Live check 2026-09-13 19:21 UTC: reaction seen by Jason ("test is successful") and recorded in the turn record. Iteration 1 closed. +- Service unit (iteration 2, Jason: "proceed as suggested", 2026-09-13). + `scripts/discord-service.sh` renders and installs a systemd user unit, + `mosaic-discord@`, from a template in `packages/discord/systemd/`. + The new `recover` policy runs first inside `run --supervised`, the unit's + main process: it clears a lock whose owner is gone and removes only the + STOP it wrote for that; every operator STOP and every held binding + refuses with the new exit code 3, which the unit never retries. First + cut used an `ExecStartPre`; systemd only honours the never-retry status + from the main process, so that looped, and the live check caught it. `systemctl --user stop` is the soft stop (SIGTERM, no STOP, + restartable); `scripts/discord.sh stop` stays the brake. This ends the + hand-removal of STOP after every rerun that step 3 of the pilot showed. + Not done: a binding reload without a restart (a ceiling or channel change + still needs stop and start). Next: the control board row. +- Live check 2026-09-13 19:35 to 19:40 UTC: Sage seat migrated to the unit; SIGKILL recovered in 16 s with the dead lock cleared; `discord.sh stop` held (start exits 3, no restart); released and READY. Iteration 2 closed. diff --git a/docs/plans/CURRENT.md b/docs/plans/CURRENT.md index 35f89cac..887b6aea 100644 --- a/docs/plans/CURRENT.md +++ b/docs/plans/CURRENT.md @@ -365,3 +365,4 @@ git history + Gitea issues. - 2026-09-03 — M19 correction: auth ownership moved to the data root (#48, owner direction — the stack never writes to default harness config locations; ROADMAP standing decision); auth.sh config-driven, accounts at /auth, 0600 enforced; test-auth 13 → 15 - 2026-09-03 — harness/provider/auth registry specification drafted (#49): agent.json harness declaration, central provider/account/settings registries, runtime seat selection, mechanical per-harness materialization, centralized OAuth refresh, Ollama endpoints, CLI contract; implementation blocked pending ten-gate review - 2026-09-13 — Discord connector pilot for the Sage seat (#1509; QUEUE rows 14–15; brief `2026-09-13_discord-connector-pilot.md`): nine review rounds with rev-code-02, live pilot steps 1–8 with private receipts, Gate H passed (Jason: the replies read as Sage). Commits 786e379c, 788515dc (pushed). MVP iteration 1, eyes reaction as a read receipt, committed 93d6b624 (local); live check pending. Connector stays up in tmux `discord-sage`; binding and token live outside the repo. +- 2026-09-13 — Discord connector iteration 2 (#1509, QUEUE row 17): systemd user service `mosaic-discord@` with a supervised run that clears a dead lock and never retries a brake (exit 3). Sage seat now runs under systemd, not tmux. Next: control board row. diff --git a/docs/plans/QUEUE.md b/docs/plans/QUEUE.md index 4e042b13..d9013c5a 100644 --- a/docs/plans/QUEUE.md +++ b/docs/plans/QUEUE.md @@ -41,6 +41,7 @@ Gaps found while working go to `docs/plans/DEFERRED.md`, not here. | 13 | Ledger "queue" section: issue/row/seat drift printed with the weekly number | darkwing; filbert reviews | #1508 | required; after row 9 | first run Monday 2026-09-21, zero violations or every one moved same day | `2026-09-13_queue-as-data.md`, Piece E | | 14 | Discord connector pilot: Sage answers in Shared Signals (chat only, no tools, no repo writes) | coordinator; Filbert or orch-01 seat reviews | #1509 | done: rev-code-02 APPROVE 26170 (round 9), committed 786e379c; pilot steps 1-8 done with private receipts, Gate H passed (Jason, 2026-09-13: replies read as Sage); connector left running for MVP iteration | H: offline suite green, eight-step live pilot with private receipts, then Jason says the reply reads as Sage | `2026-09-13_discord-connector-pilot.md` | | 15 | Discord connector: eyes reaction on every admitted message as a read receipt (MVP iteration 1) | coordinator | #1509 | done: committed 93d6b624, live check passed 19:21 UTC (turn record receipt ok, Jason: test is successful), receipt `mvp1-read-receipt-20260913T192158Z.json` in the private evidence dir; `rest.react` best effort, reaction placed at admission before the engine runs, outcome in the turn record, no reaction on drops or refusals; `scripts/test-discord.sh` 28/28 (90 node tests) | Jason sees the reaction on a live message | `2026-09-13_discord-connector-pilot.md` section 11 | +| 17 | Discord connector: systemd user service with a supervised pre-start (`recover`, exit 3 never retried) (MVP iteration 2) | coordinator | #1509 | done: `scripts/test-discord.sh` 40/40 (95 node tests); Sage seat migrated 19:35 UTC, SIGKILL recovered in 16 s with the dead lock cleared, brake held (exit 3, no restart), released and READY; receipt `mvp2-service-unit-*.json` in the private evidence dir. First cut (ExecStartPre) looped and was replaced by `run --supervised` before any traffic | the Sage connector runs under `mosaic-discord@shared-signals`, survives a kill with a clean restart, and stays down behind `discord.sh stop` | `2026-09-13_discord-connector-pilot.md` section 11 | Start message for row 6, sent from the board to darkwing: "Read docs/plans/QUEUE.md, then the plan page section "Piece 5: darkwing on @@ -79,3 +80,6 @@ Gate F or when blocked." - 2026-09-13 — coordinator: row 15 added and done (Discord read receipt, Jason's request after the pilot); committed after suites green. No other row changed. - 2026-09-13 — coordinator: row 15 moved to in verification; committed 93d6b624, connector running the new code, waiting for Jason's live message to show the reaction. No other row changed. - 2026-09-13 — coordinator: row 15 done; live reaction confirmed by Jason and by the turn record. No other row changed. +- 2026-09-13 — coordinator: rows 14–15 pushed on Jason's authorization (788515dc..dc5902aa to origin/refactor). No other row changed. +- 2026-09-13 — coordinator: row 17 added, in verification (Discord service unit, iteration 2 on Jason's "proceed as suggested"). No other row changed. +- 2026-09-13 — coordinator: row 17 done; the Sage connector runs under systemd, kill and brake checks passed live. No other row changed. diff --git a/packages/discord/README.md b/packages/discord/README.md index debb3034..746f76a9 100644 --- a/packages/discord/README.md +++ b/packages/discord/README.md @@ -14,9 +14,11 @@ CHAT-03 conversation controller exists. ``` scripts/discord.sh check -scripts/discord.sh run +scripts/discord.sh run [--supervised] scripts/discord.sh stop scripts/discord.sh unlock +scripts/discord.sh recover +scripts/discord-service.sh render | install | uninstall | status ``` `` names `/discord/.json`. The wrapper passes @@ -29,7 +31,8 @@ scripts/discord.sh unlock as the message-content intent not being granted in the developer portal. - `run` refuses when `STOP` exists or an unresolved delivery cannot be reconciled. Otherwise it starts pi, connects, and serves turns until - SIGTERM, SIGINT or `stop`. Run it in a tmux window; there is no service unit. + SIGTERM, SIGINT or `stop`. Run it under the service unit below, or by hand + in a tmux window. - `stop` writes `STOP` and sends SIGTERM to the process in `run.lock`, only when that pid is alive and both its start time and the boot id match the recorded ones; a reused pid, a pid from a previous boot, or a pid whose @@ -48,8 +51,54 @@ scripts/discord.sh unlock and releases itself if it is there, so nothing that starts during an unlock can hold the binding. `run` never reclaims a stale lock on its own; it refuses and names this command. Remove `STOP` to run again. +- `recover` is the supervised pre-start; `run --supervised` performs it + first, in the same process, and that is the form the service unit uses, + because systemd honours a never-retry exit status only from the main + process. It refuses, with exit 3 and + touching nothing, while `STOP` is present or the binding is held by a live + process, an alive pid whose identity cannot be verified, or an unreadable + record. A lock whose owner is gone, or that has no record, it clears the + way `unlock` does, then it removes the `STOP` it wrote for that so the run + that follows can claim. It removes only a `STOP` that consists of the one + line it wrote itself; a brake an operator wrote at any point, even during + the recovery, stays and the start is refused. Nothing automatic ever + removes an operator's `STOP`. -Exit codes: 0 ok, 1 operation failed, 2 invalid data or configuration, 4 usage. +Exit codes: 0 ok, 1 operation failed, 2 invalid data or configuration, 3 +refused by a brake (`STOP` present or the binding held; a supervisor must +not retry), 4 usage. + +## Service unit + +`scripts/discord-service.sh install` renders +`packages/discord/systemd/mosaic-discord@.service.in` with the repository +path and the directory of `node`, writes it to +`~/.config/systemd/user/mosaic-discord@.service` (temp file, then rename; +`--dir DIR` for another place, `--no-reload` to skip `daemon-reload`) and +prints the commands that follow. One instance per binding: + +``` +systemctl --user enable --now mosaic-discord@ start now and at login +systemctl --user status mosaic-discord@ +journalctl --user -u mosaic-discord@ -f the log (stderr of `run`) +systemctl --user stop mosaic-discord@ SIGTERM; the turn in flight finishes; restartable +scripts/discord.sh stop the brake: writes STOP; the unit stays down until STOP is removed +scripts/discord-service.sh status unit state, STOP, run.lock +``` + +What the unit does: `ExecStart` runs `run --supervised`, +`Restart=on-failure` with 15 seconds between tries and at most five in ten +minutes, and `RestartPreventExitStatus=3` so a brake is never retried. A +crash (any other non-zero exit, a signal, an engine that died) restarts, +and the supervised run clears the dead lock on its way in. `systemctl --user stop` +sends SIGTERM only; the connector exits 0 and no `STOP` is written, so the +next start needs no hand. The stop timeout is 3700 seconds, the largest turn +timeout a binding may set plus margin; a normal stop takes as long as the +turn in flight. The unit never reads the binding or the token; `run` does, +at runtime, as before. Surviving logout and reboot needs +`loginctl enable-linger`. `render` prints the unit without writing it; +`uninstall` refuses while an instance is active. The log goes to journald: +`run` writes no message text to stderr, only ids, counts and state. ## The binding @@ -142,5 +191,5 @@ malformed line), restart replay, stop and ceiling. ## Not in this piece Tools, repository writes, announcements, attachments, slash commands, DMs, -per-thread sessions, more than one server or seat, a service unit, a -control-board row. Section 8 of the brief keeps the list. +per-thread sessions, more than one server or seat, a control-board row. +Section 8 of the brief keeps the list. diff --git a/packages/discord/src/cli.mjs b/packages/discord/src/cli.mjs index 04fc204e..705e6fa6 100755 --- a/packages/discord/src/cli.mjs +++ b/packages/discord/src/cli.mjs @@ -1,9 +1,10 @@ #!/usr/bin/env node // Usage: // mosaic-discord check [--config PATH] [--repo PATH] -// mosaic-discord run [--config PATH] [--repo PATH] +// mosaic-discord run [--config PATH] [--repo PATH] [--supervised] // mosaic-discord stop [--config PATH] // mosaic-discord unlock [--config PATH] +// mosaic-discord recover [--config PATH] // // names /discord/.json. The repository wrapper // is scripts/discord.sh. @@ -23,8 +24,18 @@ // with unverifiable identity, or recorded in a file it cannot read. `run` never reclaims on its own, and a // claim that finds STOP after publishing releases itself, so unlock cannot // race a start. Remove STOP to run again. +// recover: the supervised pre-start. Refuses while STOP is present or the +// binding is held by a live or unverifiable process; clears a lock whose +// owner is gone the way unlock does, then removes the STOP it wrote for +// that, so the run that follows can claim. Never removes a STOP an operator +// wrote. `run --supervised` does the same first thing itself; the service +// unit (scripts/discord-service.sh) uses that form, because systemd only +// honours a never-retry exit status from the main process, not from a +// pre-start command. // -// Exit codes: 0 ok; 1 operation failed; 2 invalid data or configuration; 4 usage. +// Exit codes: 0 ok; 1 operation failed; 2 invalid data or configuration; +// 3 refused by a brake (STOP present or the binding held; a supervisor must +// not retry); 4 usage. import { existsSync, mkdirSync, mkdtempSync, writeFileSync, statSync, readdirSync } from "node:fs"; import { join, resolve } from "node:path"; @@ -36,17 +47,18 @@ import { createGateway, CONNECTOR_INTENTS } from "./gateway.mjs"; import { createEngine, buildPiArgs } from "./engine-pi.mjs"; import { assembleContext } from "./context.mjs"; import { createConnector } from "./connector.mjs"; -import { ensureJournal, requestStop, stopRequested, readPid, stopTarget, writePid, clearPid, unlock } from "./journal.mjs"; +import { ensureJournal, requestStop, stopRequested, readPid, stopTarget, writePid, clearPid, unlock, recover, BRAKE_EXIT } from "./journal.mjs"; const USAGE = [ "usage: mosaic-discord check [--config PATH] [--repo PATH]", - " mosaic-discord run [--config PATH] [--repo PATH]", + " mosaic-discord run [--config PATH] [--repo PATH] [--supervised]", " mosaic-discord stop [--config PATH]", " mosaic-discord unlock [--config PATH]", + " mosaic-discord recover [--config PATH]", ].join("\n"); function parse(argv) { - const opts = { command: null, binding: null, config: defaultConfigPath(), repo: process.cwd() }; + const opts = { command: null, binding: null, config: defaultConfigPath(), repo: process.cwd(), supervised: false }; for (let i = 0; i < argv.length; i++) { const a = argv[i]; if (a === "--config" || a === "--repo") { @@ -54,14 +66,17 @@ function parse(argv) { opts[a.slice(2)] = resolve(argv[++i]); } else if (a === "--help" || a === "-h") { opts.command = "help"; + } else if (a === "--supervised") { + opts.supervised = true; } else if (a.startsWith("--")) throw new DiscordError(`unknown argument: ${a}\n${USAGE}`, 4); else if (opts.command === null) opts.command = a; else if (opts.binding === null) opts.binding = a; else throw new DiscordError(`unexpected argument: ${a}\n${USAGE}`, 4); } if (opts.command === "help") return opts; - if (!["check", "run", "stop", "unlock"].includes(opts.command)) throw new DiscordError(USAGE, 4); + if (!["check", "run", "stop", "unlock", "recover"].includes(opts.command)) throw new DiscordError(USAGE, 4); if (opts.binding === null) throw new DiscordError(`${opts.command} needs a binding name\n${USAGE}`, 4); + if (opts.supervised && opts.command !== "run") throw new DiscordError(`--supervised applies to run only\n${USAGE}`, 4); return opts; } @@ -136,7 +151,11 @@ async function run(opts) { const { binding, contextFiles, pi, journalDir, sessionDir } = prepare(opts); const token = readToken(binding); ensureJournal(journalDir); - if (stopRequested(journalDir)) throw new DiscordError(`STOP is present in ${journalDir}; remove it to run`, 1); + if (opts.supervised) { + const outcome = recover(journalDir); + if (outcome === "cleared") warn("supervised start: run.lock left by a process that is gone was removed"); + } + if (stopRequested(journalDir)) throw new DiscordError(`STOP is present in ${journalDir}; remove it to run`, BRAKE_EXIT); writePid(journalDir, process.pid); const cleanupPid = () => clearPid(journalDir, process.pid); try { @@ -242,6 +261,16 @@ function unlockCommand(opts) { say("remove STOP to run again"); } +function recoverCommand(opts) { + const dataRoot = loadDataRoot(opts.config); + const binding = loadBinding(bindingPath(dataRoot, opts.binding)); + const journalDir = bindingDataDir(dataRoot, binding.name); + ensureJournal(journalDir); + const outcome = recover(journalDir); + if (outcome === "cleared") say("run.lock left by a process that is gone was removed; STOP is absent; ready to run"); + else say("no lock and no STOP; ready to run"); +} + async function main() { const opts = parse(process.argv.slice(2)); if (opts.command === "help") { @@ -251,6 +280,7 @@ async function main() { if (opts.command === "check") await check(opts); else if (opts.command === "run") await run(opts); else if (opts.command === "unlock") unlockCommand(opts); + else if (opts.command === "recover") recoverCommand(opts); else stop(opts); return opts.command === "run" ? null : 0; } diff --git a/packages/discord/src/errors.mjs b/packages/discord/src/errors.mjs index 3ba4f37f..f15f1247 100644 --- a/packages/discord/src/errors.mjs +++ b/packages/discord/src/errors.mjs @@ -1,5 +1,6 @@ // One error class for the package. exitCode follows docs/TOOLS.md: 1 operation -// failed, 2 invalid data or configuration, 4 usage. +// failed, 2 invalid data or configuration, 3 refused by a brake (STOP or a +// held binding; a supervisor must not retry), 4 usage. export class DiscordError extends Error { constructor(message, exitCode = 2, details = undefined) { super(message); diff --git a/packages/discord/src/journal.mjs b/packages/discord/src/journal.mjs index ae692f4f..4a361c85 100644 --- a/packages/discord/src/journal.mjs +++ b/packages/discord/src/journal.mjs @@ -6,12 +6,15 @@ // drops.jsonl one counter line per dropped or refused inbound message // admissions.jsonl one line per turn admitted, before the engine is asked // turns/.json one write-once record per turn -// STOP presence refuses new turns +// STOP presence refuses new turns; one JSON line per writer +// ({at, reason}), appended, so the last line names who +// braked; `recover` removes only a STOP it wrote itself // notices.jsonl once-per-day fixed lines already attempted (ceiling) // run.lock/ ownership directory (mkdir is atomic) holding owner.json // {pid, start, boot}; `stop` signals only a live pid whose // start time and boot id match; a stale lock refuses `run` -// until `unlock`, which is gated by STOP +// until `unlock`, which is gated by STOP; `recover` is the +// supervised form for a service unit's pre-start // Directories are 0700, files 0600. Lines are appended, never rewritten. import { @@ -185,6 +188,21 @@ export function clearStop(dir) { if (existsSync(path)) unlinkSync(path); } +// The STOP lines as written, oldest first. A line that does not parse is +// kept as {reason: null}: it was not written by this code, so it is never +// treated as ours. +export function readStop(dir) { + if (!stopRequested(dir)) return []; + return readFileSync(stopPath(dir), "utf8").split("\n").filter((l) => l.trim() !== "").map((l) => { + try { + const v = JSON.parse(l); + return v && typeof v === "object" && typeof v.reason === "string" ? v : { reason: null }; + } catch { + return { reason: null }; + } + }); +} + // --- run lock --- // One directory, /run.lock, is the ownership primitive: mkdir is atomic, // so two starts cannot both create it. The owner record is published inside @@ -308,17 +326,23 @@ export function ownerAlive(rec, opts) { return ownerState(rec, opts) === "live"; } +// Exit code for a refusal that a supervisor must not retry: STOP is present +// or the binding is held. Distinct from 1 (operation failed) so a service +// unit can restart after a crash and stay down after a brake. +export const BRAKE_EXIT = 3; +export const RECOVER_REASON = "recover"; + export const UNLOCK_HINT = "if no connector is running for this binding, run `scripts/discord.sh unlock `"; // Explains why an existing lock refuses a new claim. Always a DiscordError. function lockRefusal(dir, opts) { const existing = readPid(dir); const state = ownerState(existing, opts); - if (state === "absent") return new DiscordError(`run.lock exists without an owner record: a start is in progress or was interrupted; ${UNLOCK_HINT}`, 1); - if (state === "invalid") return new DiscordError(`run.lock has an owner record that cannot be read; refusing. Inspect ${ownerPath(dir)} by hand`, 1); - if (state === "live") return new DiscordError(`another connector is running for this binding (pid ${existing.pid})`, 1); - if (state === "unknown") return new DiscordError(`run.lock belongs to pid ${existing.pid}, which is alive but whose identity cannot be verified; refusing`, 1); - return new DiscordError(`run.lock belongs to pid ${existing.pid}, which is gone or is a different process now; ${UNLOCK_HINT}`, 1); + if (state === "absent") return new DiscordError(`run.lock exists without an owner record: a start is in progress or was interrupted; ${UNLOCK_HINT}`, BRAKE_EXIT); + if (state === "invalid") return new DiscordError(`run.lock has an owner record that cannot be read; refusing. Inspect ${ownerPath(dir)} by hand`, BRAKE_EXIT); + if (state === "live") return new DiscordError(`another connector is running for this binding (pid ${existing.pid})`, BRAKE_EXIT); + if (state === "unknown") return new DiscordError(`run.lock belongs to pid ${existing.pid}, which is alive but whose identity cannot be verified; refusing`, BRAKE_EXIT); + return new DiscordError(`run.lock belongs to pid ${existing.pid}, which is gone or is a different process now; ${UNLOCK_HINT}`, BRAKE_EXIT); } export function writePid(dir, pid, { now = Date.now(), identity = identityOf } = {}) { @@ -338,7 +362,7 @@ export function writePid(dir, pid, { now = Date.now(), identity = identityOf } = // this claim must not stand, however it interleaved with an unlock. if (stopRequested(dir)) { clearPid(dir, pid); - throw new DiscordError(`STOP is present in ${dir}; remove it to run`, 1); + throw new DiscordError(`STOP is present in ${dir}; remove it to run`, BRAKE_EXIT); } } @@ -348,9 +372,10 @@ export function writePid(dir, pid, { now = Date.now(), identity = identityOf } = // Refuses an owner record it cannot read. Otherwise removes the lock. // Returns the record that was cleared (null for a lock without one), or // false when there was no lock. STOP stays in place; -// remove it to run again. `beforeRemove` and `identity` are test seams. -export function unlock(dir, { beforeRemove = null, identity = identityOf } = {}) { - requestStop(dir, "unlock"); +// remove it to run again. `beforeRemove` and `identity` are test seams; +// `reason` is what the STOP line says (`recover` uses its own). +export function unlock(dir, { beforeRemove = null, identity = identityOf, reason = "unlock" } = {}) { + requestStop(dir, reason); const lock = lockPath(dir); if (!existsSync(lock)) return false; const rec = readPid(dir); @@ -363,6 +388,41 @@ export function unlock(dir, { beforeRemove = null, identity = identityOf } = {}) return rec; } +// Supervised pre-start, for the service unit's ExecStartPre. Lets a run +// that crashed (or a reboot) start again without a hand, while every +// operator brake still holds. In order: +// STOP present refuse with BRAKE_EXIT and touch nothing. `stop` +// and `unlock` wrote it; nothing automatic removes it. +// The one exception is a STOP made of exactly one +// line that an earlier `recover` wrote and then died +// before removing: that one is ours and goes. +// no run.lock nothing to do. +// owner live refuse: another connector holds the binding. +// owner unknown, or record invalid refuse, as `unlock` would. +// owner dead, or no record `unlock` (STOP written first, so a +// claim that publishes meanwhile releases itself), +// then that STOP is removed again, but only if it +// still consists of the single line `unlock` wrote +// for us. Any other line means an operator braked in +// the meantime; STOP stays and the start is refused. +// Returns "clean" or "cleared". Every refusal is a DiscordError with +// BRAKE_EXIT so a supervisor does not retry it. `beforeRemove` and +// `identity` are test seams. +export function recover(dir, { identity = identityOf, beforeRemove = null } = {}) { + const ours = (lines) => lines.length === 1 && lines[0].reason === RECOVER_REASON; + if (stopRequested(dir)) { + if (!ours(readStop(dir))) throw new DiscordError(`STOP is present in ${dir}; the brake is on. Remove it to run`, BRAKE_EXIT); + clearStop(dir); + } + if (!existsSync(lockPath(dir))) return "clean"; + const state = ownerState(readPid(dir), { identity }); + if (state === "live" || state === "unknown" || state === "invalid") throw lockRefusal(dir, { identity }); + unlock(dir, { identity, reason: RECOVER_REASON, beforeRemove }); + if (!ours(readStop(dir))) throw new DiscordError(`STOP was written by an operator while recovering ${dir}; the brake stays. Remove it to run`, BRAKE_EXIT); + clearStop(dir); + return "cleared"; +} + // The verified live owner to signal, or null. Never returns a pid whose // identity cannot be proven. export function stopTarget(dir, opts) { diff --git a/packages/discord/systemd/mosaic-discord@.service.in b/packages/discord/systemd/mosaic-discord@.service.in new file mode 100644 index 00000000..ec8c6fed --- /dev/null +++ b/packages/discord/systemd/mosaic-discord@.service.in @@ -0,0 +1,35 @@ +# Mosaic Discord connector, one instance per binding: mosaic-discord@. +# Rendered by scripts/discord-service.sh from packages/discord/systemd/; +# @REPO@ and @PATH@ are filled in at install time. Edit the template and +# reinstall; do not edit the installed copy. +[Unit] +Description=Mosaic Discord connector (%i) +Documentation=file://@REPO@/packages/discord/README.md +After=network-online.target +Wants=network-online.target +StartLimitIntervalSec=600 +StartLimitBurst=5 + +[Service] +Type=simple +WorkingDirectory=@REPO@ +Environment=PATH=@PATH@ +# A supervised run first clears a lock left by a crash or a reboot, and +# refuses with exit 3 while STOP is present or the binding is held. Exit 3 +# is never retried: the brake belongs to the operator. This happens inside +# the main process on purpose: systemd honours RestartPreventExitStatus only +# for the main process, so a refusing ExecStartPre would loop. +ExecStart=@REPO@/scripts/discord.sh run %i --supervised +Restart=on-failure +RestartSec=15 +RestartPreventExitStatus=3 +# `systemctl --user stop` sends SIGTERM; the connector finishes the turn in +# flight, then exits 0. The stop timeout covers the largest turn timeout a +# binding may set (3600 s) plus margin; a normal stop takes seconds. +KillSignal=SIGTERM +KillMode=mixed +TimeoutStopSec=3700 +NoNewPrivileges=yes + +[Install] +WantedBy=default.target diff --git a/packages/discord/tests/binding.test.mjs b/packages/discord/tests/binding.test.mjs index a39b991a..e3bb0ba9 100644 --- a/packages/discord/tests/binding.test.mjs +++ b/packages/discord/tests/binding.test.mjs @@ -137,6 +137,6 @@ test("cli: run refuses when STOP is present, before any network use", () => { assert.equal(r0.status, 0, r0.stderr); assert.match(r0.stdout, /STOP written/); const r = runCli(["run", "test-seat", "--config", dep.config, "--repo", repo]); - assert.equal(r.status, 1, r.stderr); + assert.equal(r.status, 3, r.stderr); assert.match(r.stderr, /STOP is present/); }); diff --git a/packages/discord/tests/recover.test.mjs b/packages/discord/tests/recover.test.mjs new file mode 100644 index 00000000..45aa67a9 --- /dev/null +++ b/packages/discord/tests/recover.test.mjs @@ -0,0 +1,192 @@ +// `recover`: the supervised pre-start behind the service unit. It may clear +// a lock whose owner is gone and remove only the STOP it wrote for that; +// every operator brake, and every held binding, refuses with exit 3. +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { writeFileSync, existsSync, mkdirSync, readFileSync, unlinkSync } from "node:fs"; +import { join } from "node:path"; +import { spawn, spawnSync } from "node:child_process"; +import { fileURLToPath } from "node:url"; +import { + ensureJournal, recover, requestStop, readStop, stopRequested, stopPath, lockPath, ownerPath, writePid, BRAKE_EXIT, + RECOVER_REASON, +} from "../src/journal.mjs"; +import { DiscordError } from "../src/errors.mjs"; +import { makeRoot, makeRepo, makeDeployment } from "./helpers.mjs"; + +const cli = fileURLToPath(new URL("../src/cli.mjs", import.meta.url)); +const worker = fileURLToPath(new URL("../fixtures/legacy-owner-worker.mjs", import.meta.url)); +const BOOT = "01234567-89ab-cdef-0123-456789abcdef"; + +function journal() { + const dir = join(makeRoot(), "journal"); + ensureJournal(dir); + return dir; +} + +function deadPid() { + const r = spawnSync(process.execPath, ["-e", "process.stdout.write(String(process.pid))"], { encoding: "utf8" }); + return Number(r.stdout); +} + +function publish(dir, rec) { + mkdirSync(lockPath(dir), { recursive: true }); + writeFileSync(ownerPath(dir), JSON.stringify(rec) + "\n", { mode: 0o600 }); +} + +const brake = (re) => (err) => err instanceof DiscordError && err.exitCode === BRAKE_EXIT && re.test(err.message); + +// A live owner with the given identity arguments, held until `release`. +function holdLock(dir, startArg, bootArg) { + const done = join(dir, `done-${startArg}-${bootArg}`); + const child = spawn(process.execPath, [worker, dir, done, startArg, bootArg], { stdio: ["ignore", "pipe", "inherit"] }); + const published = new Promise((resolve) => { + let out = ""; + child.stdout.on("data", (d) => { out += d; if (out.includes("legacy-published")) resolve(); }); + }); + const closed = new Promise((resolve) => child.on("close", resolve)); + return { published, release: () => { writeFileSync(done, ""); return closed; } }; +} + +test("recover: nothing to do is clean; a lock whose owner is gone or that has no record is cleared and STOP ends up absent", () => { + const dir = journal(); + assert.equal(recover(dir), "clean"); + assert.equal(stopRequested(dir), false); + + publish(dir, { pid: deadPid(), start: "1", boot: BOOT, at: "x" }); + assert.equal(recover(dir), "cleared"); + assert.equal(existsSync(lockPath(dir)), false, "dead owner's lock removed"); + assert.equal(stopRequested(dir), false, "the STOP written for the unlock is gone again"); + + mkdirSync(lockPath(dir)); + assert.equal(recover(dir), "cleared", "a lock without a record (interrupted start) is cleared too"); + assert.equal(existsSync(lockPath(dir)), false); + assert.equal(stopRequested(dir), false); + assert.equal(recover(dir), "clean"); +}); + +test("recover: an operator STOP refuses with exit 3 and is never removed, whatever the lock says", () => { + const dir = journal(); + requestStop(dir, "cli stop"); + assert.throws(() => recover(dir), brake(/STOP is present/)); + assert.equal(stopRequested(dir), true); + publish(dir, { pid: deadPid(), start: "1", boot: BOOT, at: "x" }); + assert.throws(() => recover(dir), brake(/STOP is present/)); + assert.equal(existsSync(lockPath(dir)), true, "the lock is not touched behind a brake"); + assert.equal(readStop(dir).length, 1, "nothing appended to STOP"); + + // A STOP that recover itself wrote and then failed to remove is ours, but + // only when it is the whole file. + const own = journal(); + requestStop(own, RECOVER_REASON); + assert.equal(recover(own), "clean"); + assert.equal(stopRequested(own), false); + requestStop(own, RECOVER_REASON); + requestStop(own, "unlock"); + assert.throws(() => recover(own), brake(/STOP is present/)); + assert.equal(readStop(own).length, 2); + + // A line this code did not write is never ours. + const foreign = journal(); + writeFileSync(stopPath(foreign), "stop\n", { mode: 0o600 }); + assert.throws(() => recover(foreign), brake(/STOP is present/)); + assert.deepEqual(readStop(foreign), [{ reason: null }]); +}); + +test("recover: a brake written during the unlock wins; STOP stays with both lines and the start is refused", () => { + const dir = journal(); + publish(dir, { pid: deadPid(), start: "1", boot: BOOT, at: "x" }); + assert.throws( + () => recover(dir, { beforeRemove: () => requestStop(dir, "cli stop") }), + brake(/written by an operator while recovering/), + ); + assert.equal(existsSync(lockPath(dir)), false, "the dead lock was removed before the brake was seen"); + assert.deepEqual(readStop(dir).map((l) => l.reason), [RECOVER_REASON, "cli stop"]); + assert.throws(() => recover(dir), brake(/STOP is present/), "and it stays refused afterwards"); +}); + +test("recover: a held binding refuses with exit 3 and writes no STOP: live owner, alive pid without verifiable identity, unreadable record", async () => { + const live = journal(); + const held = holdLock(live, "real", "real"); + await held.published; + try { + assert.throws(() => recover(live), brake(/another connector is running/)); + assert.equal(stopRequested(live), false, "recover never brakes a running connector"); + assert.equal(existsSync(ownerPath(live)), true); + } finally { + await held.release(); + } + + const unknown = journal(); + const legacy = holdLock(unknown, "real", "-"); + await legacy.published; + try { + assert.throws(() => recover(unknown), brake(/cannot be verified/)); + assert.equal(stopRequested(unknown), false); + assert.equal(existsSync(ownerPath(unknown)), true); + } finally { + await legacy.release(); + } + + const invalid = journal(); + mkdirSync(lockPath(invalid)); + writeFileSync(ownerPath(invalid), "{not json", { mode: 0o600 }); + assert.throws(() => recover(invalid), brake(/cannot be read/)); + assert.equal(stopRequested(invalid), false); + assert.equal(readFileSync(ownerPath(invalid), "utf8"), "{not json"); + + // Our own live claim is a held binding too. + const mine = journal(); + writePid(mine, process.pid); + assert.throws(() => recover(mine), brake(/another connector is running/)); + assert.equal(stopRequested(mine), false); +}); + +test("cli: recover exits 0 when ready, 3 behind a brake or a held binding, and run's own STOP refusal is 3", () => { + const root = makeRoot(); + const repo = makeRepo(root); + const dep = makeDeployment(root); + const run = (args) => spawnSync(process.execPath, [cli, ...args, "--config", dep.config, "--repo", repo], { encoding: "utf8" }); + let r = run(["recover", "test-seat"]); + assert.equal(r.status, 0, r.stderr); + assert.match(r.stdout, /ready to run/); + + r = run(["stop", "test-seat"]); + assert.equal(r.status, 0, r.stderr); + r = run(["recover", "test-seat"]); + assert.equal(r.status, 3, r.stderr); + assert.match(r.stderr, /the brake is on/); + r = run(["run", "test-seat"]); + assert.equal(r.status, 3, r.stderr); + + const journalDir = join(dep.dataRoot, "discord", "test-seat"); + unlinkSync(stopPath(journalDir)); + publish(journalDir, { pid: deadPid(), start: "1", boot: BOOT, at: "x" }); + r = run(["recover", "test-seat"]); + assert.equal(r.status, 0, r.stderr); + assert.match(r.stdout, /was removed; STOP is absent/); + assert.equal(existsSync(lockPath(journalDir)), false); + assert.equal(stopRequested(journalDir), false); + + r = run(["recover"]); + assert.equal(r.status, 4); + + // The service unit's form: the same policy inside the main process, so + // the exit status is the one systemd never retries. + r = run(["stop", "test-seat"]); + assert.equal(r.status, 0, r.stderr); + r = run(["run", "test-seat", "--supervised"]); + assert.equal(r.status, 3, r.stderr); + assert.match(r.stderr, /the brake is on/); + assert.equal(stopRequested(journalDir), true, "the operator's STOP stays"); + unlinkSync(stopPath(journalDir)); + publish(journalDir, { pid: process.pid, at: "x" }); + r = run(["run", "test-seat", "--supervised"]); + assert.equal(r.status, 3, r.stderr); + assert.match(r.stderr, /cannot be verified/); + assert.equal(existsSync(ownerPath(journalDir)), true, "a held binding is left alone"); + assert.equal(stopRequested(journalDir), false); + r = run(["stop", "test-seat", "--supervised"]); + assert.equal(r.status, 4); +}); + diff --git a/scripts/discord-service.sh b/scripts/discord-service.sh new file mode 100755 index 00000000..7f9bca17 --- /dev/null +++ b/scripts/discord-service.sh @@ -0,0 +1,139 @@ +#!/usr/bin/env bash +# `scripts/discord-service.sh render|install|uninstall|status []`: +# the systemd user unit that supervises `scripts/discord.sh run `. +# The template is packages/discord/systemd/mosaic-discord@.service.in; the +# rendered unit is mosaic-discord@.service in the systemd user directory, one +# instance per binding (mosaic-discord@). Nothing here reads the +# binding or the token. See packages/discord/README.md, "Service unit". +# +# render print the rendered unit on stdout +# install [--dir DIR] [--no-reload] +# write the unit (write to a temp file, then +# rename) and run `systemctl --user daemon-reload` +# uninstall [--dir DIR] [--no-reload] +# remove the unit; refuses while an instance is active +# status unit state, STOP and run.lock for one binding +# +# Exit codes: 0 ok, 1 operation failed, 4 usage. +set -euo pipefail +REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +TEMPLATE="$REPO/packages/discord/systemd/mosaic-discord@.service.in" +UNIT_NAME="mosaic-discord@.service" +USAGE="usage: scripts/discord-service.sh render | install [--dir DIR] [--no-reload] | uninstall [--dir DIR] [--no-reload] | status " + +die() { echo "discord-service: $*" >&2; exit 1; } +usage() { echo "$USAGE" >&2; exit 4; } + +render() { + [ -f "$TEMPLATE" ] || die "template missing: $TEMPLATE" + local node_dir + node_dir="$(dirname "$(command -v node || true)")" + [ -n "$node_dir" ] && [ "$node_dir" != "." ] || die "node not found on PATH" + local path="/usr/local/bin:/usr/bin:/bin" + case ":$path:" in *":$node_dir:"*) ;; *) path="$node_dir:$path" ;; esac + case "$REPO" in *@*|*'|'*) die "repository path contains a character the template cannot carry: $REPO" ;; esac + sed -e "s|@REPO@|$REPO|g" -e "s|@PATH@|$path|g" "$TEMPLATE" +} + +# Parse --dir and --no-reload for install and uninstall. +UNIT_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user" +RELOAD=1 +parse_flags() { + while [ $# -gt 0 ]; do + case "$1" in + --dir) [ $# -ge 2 ] || usage; UNIT_DIR="$2"; shift 2 ;; + --no-reload) RELOAD=0; shift ;; + *) usage ;; + esac + done +} + +install_unit() { + parse_flags "$@" + mkdir -p "$UNIT_DIR" + local target="$UNIT_DIR/$UNIT_NAME" tmp + tmp="$(mktemp "$UNIT_DIR/.$UNIT_NAME.XXXXXX")" + render > "$tmp" + chmod 0644 "$tmp" + if [ -f "$target" ] && cmp -s "$tmp" "$target"; then + rm -f "$tmp" + echo "unchanged: $target" + else + mv -f "$tmp" "$target" + echo "written: $target" + fi + if [ "$RELOAD" = 1 ]; then + systemctl --user daemon-reload || die "daemon-reload failed" + echo "daemon-reload done" + fi + cat < start now and at login + systemctl --user status mosaic-discord@ + journalctl --user -u mosaic-discord@ -f the connector's log + systemctl --user stop mosaic-discord@ SIGTERM; restartable + scripts/discord.sh stop the brake: writes STOP, the unit stays down until STOP is removed +survive logout and reboot only with lingering on: loginctl enable-linger $USER +MSG +} + +uninstall_unit() { + parse_flags "$@" + local target="$UNIT_DIR/$UNIT_NAME" + if [ "$RELOAD" = 1 ]; then + local active + active="$(systemctl --user list-units --no-legend --plain 'mosaic-discord@*' 2>/dev/null | awk '$3 == "active" || $3 == "activating" || $3 == "deactivating" {print $1}')" + [ -z "$active" ] || die "refusing: instance(s) still running: $(echo "$active" | tr '\n' ' ')stop them first" + fi + if [ -f "$target" ]; then + rm -f "$target" + echo "removed: $target" + else + echo "absent: $target" + fi + if [ "$RELOAD" = 1 ]; then + systemctl --user daemon-reload || die "daemon-reload failed" + echo "daemon-reload done" + fi +} + +status_unit() { + [ $# -eq 1 ] || usage + local binding="$1" + case "$binding" in ''|*[!A-Za-z0-9_-]*) die "binding name must be [A-Za-z0-9_-]+" ;; esac + local unit="mosaic-discord@$binding.service" + echo "unit: $unit" + echo " enabled: $(systemctl --user is-enabled "$unit" 2>&1 || true)" + echo " active: $(systemctl --user is-active "$unit" 2>&1 || true)" + systemctl --user show "$unit" -p MainPID -p NRestarts -p ExecMainStartTimestamp -p Result 2>/dev/null | sed 's/^/ /' + local config="${XDG_CONFIG_HOME:-$HOME/.config}/mosaic-dev/config.json" + local data_root + data_root="$(node -e 'const c=JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")); process.stdout.write(String(c.dataRoot||""))' "$config" 2>/dev/null || true)" + if [ -z "$data_root" ]; then + echo " journal: dataRoot not readable from $config" + return 0 + fi + data_root="${data_root/#\~/$HOME}" + local dir="$data_root/discord/$binding" + if [ -f "$dir/STOP" ]; then + echo " STOP: present ($(tail -n 1 "$dir/STOP"))" + else + echo " STOP: absent" + fi + if [ -d "$dir/run.lock" ]; then + echo " run.lock: present$( [ -f "$dir/run.lock/owner.json" ] && printf ' (owner pid %s)' "$(node -e 'process.stdout.write(String(JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")).pid))' "$dir/run.lock/owner.json" 2>/dev/null || echo '?')" )" + else + echo " run.lock: absent" + fi +} + +[ $# -ge 1 ] || usage +cmd="$1"; shift +case "$cmd" in + render) [ $# -eq 0 ] || usage; render ;; + install) install_unit "$@" ;; + uninstall) uninstall_unit "$@" ;; + status) status_unit "$@" ;; + -h|--help) echo "$USAGE" ;; + *) usage ;; +esac diff --git a/scripts/test-discord.sh b/scripts/test-discord.sh index be44d4db..3a7c3b6c 100755 --- a/scripts/test-discord.sh +++ b/scripts/test-discord.sh @@ -25,7 +25,7 @@ echo "toolchain: node $(node --version)" echo # --- syntax --- -for f in packages/discord/src/*.mjs packages/discord/tests/*.mjs scripts/discord.sh; do +for f in packages/discord/src/*.mjs packages/discord/tests/*.mjs packages/discord/fixtures/*.mjs scripts/discord.sh scripts/discord-service.sh; do case "$f" in *.sh) bash -n "$f" >/dev/null 2>&1 ;; *) node --check "$f" >/dev/null 2>&1 ;; @@ -71,6 +71,44 @@ check "scripts/discord.sh --help exits 0" $? scripts/discord.sh check >/dev/null 2>&1 [ $? -eq 4 ] check "scripts/discord.sh check without a binding exits 4" $? +scripts/discord.sh recover >/dev/null 2>&1 +[ $? -eq 4 ] +check "scripts/discord.sh recover without a binding exits 4" $? + +# --- the service unit: rendered from the template, never touching systemd here --- +UNITS="$SANDBOX/units" +scripts/discord-service.sh >/dev/null 2>&1 +[ $? -eq 4 ] +check "scripts/discord-service.sh without a command exits 4" $? +REPO_DIR="$(pwd)" +scripts/discord-service.sh render >"$SANDBOX/unit.rendered" 2>/dev/null \ + && grep -qF "ExecStart=$REPO_DIR/scripts/discord.sh run %i --supervised" "$SANDBOX/unit.rendered" \ + && ! grep -q '^ExecStartPre=' "$SANDBOX/unit.rendered" \ + && grep -qx 'RestartPreventExitStatus=3' "$SANDBOX/unit.rendered" \ + && grep -qx 'Restart=on-failure' "$SANDBOX/unit.rendered" \ + && grep -qx 'KillSignal=SIGTERM' "$SANDBOX/unit.rendered" \ + && ! grep -q '@REPO@\|@PATH@' "$SANDBOX/unit.rendered" +check "service unit renders with the repository path, a supervised run as the main process, and exit 3 never retried" $? +scripts/discord-service.sh install --dir "$UNITS" --no-reload >"$SANDBOX/install.1" 2>&1 \ + && [ -f "$UNITS/mosaic-discord@.service" ] \ + && grep -q '^written:' "$SANDBOX/install.1" \ + && cmp -s "$UNITS/mosaic-discord@.service" "$SANDBOX/unit.rendered" \ + && [ "$(stat -c %a "$UNITS/mosaic-discord@.service")" = "644" ] \ + && [ -z "$(ls -A "$UNITS" | grep -v '^mosaic-discord@.service$')" ] +check "service install writes the rendered unit (0644) and leaves no temp file" $? +scripts/discord-service.sh install --dir "$UNITS" --no-reload >"$SANDBOX/install.2" 2>&1 && grep -q '^unchanged:' "$SANDBOX/install.2" +check "service install a second time reports unchanged" $? +if command -v systemd-analyze >/dev/null 2>&1; then + cp "$UNITS/mosaic-discord@.service" "$UNITS/mosaic-discord@sample.service" + systemd-analyze --user verify "$UNITS/mosaic-discord@sample.service" >"$SANDBOX/verify.log" 2>&1 && ! grep -qi 'warning\|error\|fail' "$SANDBOX/verify.log" + check "systemd-analyze verify accepts the rendered unit" $? + rm -f "$UNITS/mosaic-discord@sample.service" +fi +scripts/discord-service.sh uninstall --dir "$UNITS" --no-reload >/dev/null 2>&1 && [ ! -e "$UNITS/mosaic-discord@.service" ] +check "service uninstall removes the unit file" $? +scripts/discord-service.sh install --dir "$UNITS" --no-reload --bogus >/dev/null 2>&1 +[ $? -eq 4 ] +check "service install with an unknown flag exits 4" $? echo echo "discord suite: $PASS passed, $FAIL failed"