# Row 281 build notes: a fire or a delete revokes the instance's grants (#1786) Builder Megavolt, reviewer Dewey, gate owner Sage. Design draft d6 for Dewey's approval. No code until it is approved. The results sections are filled in after the runs on Sage's gate. History: d1 dfebd16cc. d2 c746ce378 added Sage's Q1/Q2 rulings (sage-2573). d3 36c2d98b8 added the approved-before-fire tests (sage-2574). d4 answers Dewey's CHANGES on d3 (dewey-0823): - B1: the set takes every grant decision with no outcome, and a raise refuses a fired or firing agent. - B2: step G runs detached from the verb, so neither caller waits on the network. - B3: one step G per agent, with named refusals for everything that could race it. - N1-N3 are folded in. d5 adds sage-2575: two requests per granted delete, with its five conditions. It also adds a `job` id to every answer, event and message, and tests T3, T3b and T3c with mutants M16-M18. d6 answers Dewey's CHANGES on d5 (dewey-0825): - C1: the job's step C records authority without the chief's claim (`#authorityFor`, as a landing's record does), processing-halted doesn't hold the job, and a step B or C failure records `fire.failed {job, step, code}` with its own message. Tests T11 and T12, mutants M19 and M20. - N4: boot closes a job a restart interrupted, with one message. Test T13, mutant M21. - N5: section 7 names the refusals the manual verbs gain. - N6: the set counts Q1 leftover lines, in fire step 3 and in the delete text. ## Sources - Brief: `docs/plans/2026-10-22_fire-revokes-grants.md`, "Brief: no grant outlives its instance" (sage-2568, mos-1271, Vikunja 524). - Code read at 47a35bf43: `packages/bus/src/broker.mjs`, `runtime.mjs`, `credentials.mjs`, `host-grant.mjs`, `gitea-grant.mjs`, `packages/business/src/instance-settings.mjs`, `tools/host/mosaic-keys-write`. - Row 256's delete (honker/row256-r2 at 1e761962a, not landed) changes only the delete's checks. Its write still goes through `lifecycleDelete` and the same `revoke()`, so this design covers it unchanged. - Row 235 r6 section 6 (host revoke) and #436b r5 section 6 (Gitea revoke and the fire steps). ## 1. Why: what happens today (read from the code, to be pinned by a test before the change) In short: today a delete of a granted instance unlinks the key files but leaves both grant records, the host lines and the Gitea token live, and a fire refuses `credential-revoke-failed` (sage-2573 asked for this here and on #1786). The details: - **Fire.** `revoke()` (runtime.mjs:151) revokes only the roles entry's `credentials` services, then lists the agent's store directory and refuses `credential-revoke-failed` if any file is left. A host grant's key is `///host.` (host-grant.mjs:102), and a Gitea grant's token is `.../gitea` (gitea-grant.mjs:379). Neither is a roles-entry service, so a fire of an instance with a grant refuses at step B with "host. still in the store". It doesn't leak, but it can't finish until a human runs `mosaic hosts revoke` or `mosaic grants revoke` by hand. If the business file also names `gitea` as a roles-entry credential, `credentials.revoke(instance, 'gitea')` unlinks the granted token file with no `gitea_grants` change and no Gitea call. - **Delete.** `credentials.remove(instance)` (credentials.mjs:400) revokes every registered reference, `host.` and `gitea` included, and unlinks the directory. The `host_grants` and `gitea_grants` records stay, the host line stays on each host, and the Gitea token, user and login stay live. This is the leak. The next start then finds a grant record whose key file is gone. What the start does with that isn't read yet. The T1 test below pins it. - **An open grant decision.** A fire delists the instance but keeps its roles entry. An approved host or Gitea grant decision for that agent that hasn't run yet still passes `#hostDefinition` and `#giteaRole`, so it can still apply after the fire. A delete removes the roles entry, so its executor refuses `unknown-role` or `gitea-not-declared`. - **A raise after the fire** (dewey-0823 B1). `#raiseGrant` (broker.mjs:1529) checks only that the caller is the chief, and the grant bindings check only the roles entry. So the chief can raise a new grant for a fired agent, and if the human approves it, it applies. ## 2. Where the grant revokes sit (D1, rewritten in d4 for B2) d3 awaited step G inside the verb. Dewey's B2 showed that neither caller can wait that long: - **Delete.** The launcher's `host.op` waits `REQUEST_TIMEOUT_MS = 10000` (cli/src/host.mjs:31). On a timeout, `breakChannel` stops the whole bus host. One SSH remove can take 30 s (host-grant.mjs:231, :311), and each Gitea call 15 s (`CALL_MS`, gitea-grant.mjs:38). - **Fire.** server.mjs answers a verb that isn't on its async list synchronously (:154), and the async list gives 120 s (:150). The chief's runner client times out at 30 s (harness/src/runner.mjs:449). Two hosts plus the Gitea calls can pass both limits. So step G never runs inside a verb. The fire and the delete both use the same answer-at-once job, for the same timeout reason (sage-2575 condition 5). Both verbs stay synchronous and answer at once. Step G runs as tracked broker work after the answer, as row 236's grant executors do (`#track`, `setImmediate`). No caller waits on the network, so `server.mjs`, `process.mjs`, `cli/src/host.mjs` and the launcher don't change. Each job gets an id, `job` (a UUID), when it starts. The verb's answer, the job's start event, its end event and every inbox message about it carry that id. The human and the chief can then match a later outcome to the request that started it. ### The fire 1. `write('instance.fireCheck')`, as today. 2. Step A, as today, with one addition. In the same transaction, after `fire.stopped`, the grant set is read (section 3). Reading it there means a grant raise either committed before step A, and is in the set, or comes after `fire.stopped` and refuses (section 3d). 3. If the set is empty, steps B and C run synchronously, exactly as today. A fire with no grant answers today's result with today's events (T7). The set counts everything step G would touch (d6, N6): grant rows, open decisions, the agent's Gitea leftovers (section 5) and its Q1 leftover host lines (section 5). Any of them makes the set non-empty and goes to the job. 4. Otherwise the verb takes the agent's step G lock (section 3e), records `fire.revoking {job, grants}` in step A's transaction, starts the job, and answers at once: `{...stopped, job, revoking: ['host:' | 'gitea', ...], done: false}`. There's no `transaction`, since step B hasn't run. 5. The job runs step G, then step B (`write('instance.fire')`) and step C (`#recordAuthority`, below, then `fire.done {job, grants}`). It releases the lock when it ends, whichever way it ends, `fire.failed` included. What the chief sees meanwhile: the instance is "firing". That's an existing state (`#firing`, broker.mjs:645): `fire.stopped` with no later `fire.done`. Starts already refuse `instance-firing`. - A repeat fire while the job runs refuses `fire-in-progress`, a new code. - A repeat after the job finishes takes the delisted path (webby-0094 N1) and answers done. - A repeat after a refused step G runs step G again (section 5). The human sees the outcome either way: `fire.done` names the job and the grants, and a refusal sends the inbox message in section 4. **Steps B and C in the job (d6, C1).** d4 left "step C without the chief's run" as a build-time check, and Dewey checked it: it fails. - `#consumeAuthority` calls `#checkAuthority` (broker.mjs:2852). That runs `#processingWork()` and `#agent(s)`, and `#agent` refuses `not-holder` once the chief's run no longer holds its claim (:749). - After the verb answers, the job can take a minute or more. If the chief's run ends meanwhile, step C would throw after step B had delisted. The instance would be left delisted with no `fire.done`. So the verb does the full check, and the job only records: - The verb runs `#checkAuthority(s, action, authority)`, as today, before step A. The chief holds its claim then. - The job's step C runs `#authorityFor(s, action, authority)`, the check without the holder test. It then runs the `#spent` check and the `action.allowed` event, as `#consumeAuthority` does. That's the shape a landing's record already uses (`recordPublish`, broker.mjs:3174, row 137): the run that asked held its claim when it asked, and may have ended since. It's one small private method, `#recordAuthority`, shared with nothing else. - The job keeps a copy of the session fields `#authorityFor` and `#event` read (business, role, thread, run). It doesn't keep the live session object. - `processing-halted` doesn't hold the job. A halt stops agent work. Step G only removes access, and steps B and C finish a decision the chief made while processing ran. Holding them would leave a delisted or half-revoked instance "firing" for the length of the halt. The verb itself still refuses during a halt, through `#checkAuthority`, as today. **A step B or C failure in the job** gets its own record, not `grant.revoke.refused`, since every grant in the set was revoked by then: - It records `fire.failed {job, step: 'B' | 'C', code}`. `code` is the thrown code, for example the writer's `credential-revoke-failed`, or `decision-consumed`. - It sends an inbox message: > Fire of : job revoked its grants () but stopped at step (). The instance > is still firing. Run the fire again once is cleared. - The instance stays firing. A repeat fire finds an empty set and takes today's path for that state, with the chief's own claim: B and C after a step B failure (T12), the delisted path (webby-0094 N1) after a step C failure. `grant.revoke.refused` with reason `internal` stays only for an unexpected throw inside step G. ### The delete (sage-2575) `lifecycleDelete` (broker.mjs:3293) stays synchronous. After its running and starting checks, it reads the grant set. - **No grant.** An empty set: `instances.set delete`, one request, exactly as today (condition 4, T3c). - **The first request.** For a non-empty set, it takes the lock and records `grants.revoking {job, agent, verb: 'lifecycle.delete', grants}`. It starts the job and refuses `grants-revoking` at once. The launcher closes the request with that code, well inside its 10 s. The launcher can carry only the code, so the broker also sends the human an inbox message (condition 1): > Delete of is waiting on its grants. Revoking started (job ) for: host:, ..., gitea. > Raise the delete again when the inbox says the revoke is done. - **A delete raised while the job runs.** It refuses `grants-revoking` again and starts no second job (condition 2, the 3e lock). No second message goes out either: the first one still applies. - **The job ends.** The human gets one inbox message (condition 3). - If it completed, the message names each grant revoked, with any `grantsLeft` (section 5), then "Raise the delete again." It records `grants.revoked.done {job, agent, verb, grants, grantsLeft}`. - If it refused, the message names the grant, the reason and how to clear it by hand, as section 4 describes. It records `grant.revoke.refused {job, ...}`. - **The delete raised after a completed job** finds an empty set and goes as today. - **The delete raised after a refused job** doesn't complete over a live grant. Its set is non-empty, since the refused grant's record is still there. It refuses `grant-revoke-failed`, naming the earlier job's reason in a message: > Delete of refused: job couldn't revoke (). Retrying now as job . Raise > the delete again when the inbox says it's done, or clear by hand first. It starts a new job for what's left. The broker knows the last job refused because the agent's latest job end event is a `grant.revoke.refused` with verb `lifecycle.delete`, and no `grants.revoked.done` came after it. A delete after the human cleared the grant by hand doesn't find an empty set if a manual revoke left a line (Q1). The leftover line makes the set non-empty, so that delete starts a job for the line removal and refuses `grants-revoking` (d6, N6, and T3b's second case). If nothing is left, the set is empty and the delete goes on. The tradeoff (ruled by sage-2575): deleting a granted instance now always takes two requests. That keeps the cli and the host channel out of row 281, and a delete of an instance with a grant is rare. Holding the launcher's request open past its 10 s would need a host channel change and a strand path, and polling would widen the row into the launcher for a one-time wait. While a delete's step G runs, the instance is still listed and not firing, so it could be started. That's harmless: its grants are being revoked, and the repeat delete refuses `instance-running` as today. No new start check. ### Why not in `beforeWrite` or `revoke()` Both are synchronous and run inside the organization writer, after its checks. A host `remove` is SSH and a Gitea revoke is HTTP, so they can't run there. The business package also has no reach into the broker's grant tables, and shouldn't get any. ### Why after step A A fire is the decision to stop the agent. If step G came first, a host that can't be reached would keep the fired agent running. After step A, a refused step G leaves the same state as a failed step B today: the instance is "firing" (stopped, still listed) until a repeat. Row 194 already handles that state. The repeat fire's delisted path (webby-0094 N1) also runs step G, since the set is read before step B on both paths. ## 3. Step G: what it revokes, and how The set, for (business, agent), is read in one store transaction: step A's for a fire, the delete's checks for a delete (section 2). It holds: - each `host_grants` row of the agent; - each host named by an open host grant decision for the agent; - the Gitea grant if the agent has a `gitea_grants` row, a `gitea_users` row, or an open Gitea grant decision. An open decision (d4, B1) is any grant decision raised for the agent with no outcome yet. That is: not denied, not expired, no `grant.applied` or `grant.failed`, and no `grant.revoked` of the same (agent, host), or of the agent's Gitea, recorded after its raise. A raised decision the human hasn't decided is open. d3 required "approved", which left that one out. An empty set means no step G and no new records. A fire of an instance with no grant is then exactly today's fire, with no new events (brief tests, fourth bullet). ### 3a. A host grant: the host line first (D2) hosts.revoke goes local first: the record drops, the key leaves memory and its file, and only then is `remove ` sent. If the host can't be reached, the revoke closes with `hostLineLeft` and the record is already gone. The brief says a fire may not drop a grant record while its key could still be on the host. So the fire's host revoke reverses one step and keeps the rest the same: 1. Abort an executor in flight for (agent, host), with no record change (`hostKeys.abortClaim`, as the `credential-unknown` branch of `#revokeHost` already does). 2. `remove ` through the installer, pinned to the host record's keys. `mosaic-keys-write` says "a missing tag changes nothing", so this is safe to repeat and safe for a host named only by an open decision. If the host record is gone, or the remove doesn't exit 0, step G refuses for this host (section 4). Nothing local has changed for this grant: the record, the key file and the memory copy stay. 3. hosts.revoke's step 1 transaction, unchanged: the `host_grants` row goes and `grant.revoked {revoke, agent, host, tag, hostLine: 'pending', granted, reason, by}` is recorded, plus `fire: true` (or `delete: true`) so the human can tell it from a manual revoke. 4. hosts.revoke's step 2, unchanged: `hostKeys.revoke`, so the key leaves memory and its file goes. 5. `grant.revoke.closed {hostLine: 'removed'}` with no second remove. Step 2 already did it. This is one shared method, `#revokeHostGrant(business, agent, host, reason, by, { lineFirst })`. hosts.revoke calls it with `lineFirst: false` and its behavior and records don't change (brief item 4). The fire and the delete call it with `lineFirst: true`. The tradeoff: hosts.revoke goes local first so that "the part that matters can't wait on the network" (r6 section 6). For a fire, step A has already stopped the agent's runs, and a refused fire keeps the instance "firing". So waiting on the host costs nothing in exposure, and the record never lies about the host. ### 3b. The Gitea grant (D3) The Gitea revoke stays grants.revoke's steps with `fire: true`, which is what `credentials delete` already runs (#436b section 6). Step 1 takes the token out of memory and unlinks the file. Step 2 is `#giteaRevokeLocal(..., fire: true)`. Step 3 is `#giteaCloseRevoke`: every token deleted and `prohibit_login: true`. The fire awaits step 3 instead of leaving it to `setImmediate`. The Gitea side can't go remote first the way a host does. The record has to drop and `grant.revoked` has to name the tokens before the DELETE calls, or a restart in between would find a live record whose tokens are gone (section 5's reconcile). Two changes keep the fire closed: - **Preflight.** Before step 1, `#giteaAdminChecked` runs. If the admin token can't be read or checked, step G refuses with nothing changed. - **Left means not done.** If `grant.revoke.closed` lists anything in `serviceLeft`, step G refuses. The token ids stay in the evidence, which is exactly what `mosaic grants sweep gitea` retries, so nothing forgets them. The record is gone, but no key of this stack's is still usable: the file and memory copy are gone, and only the Gitea side is left. That is the case the brief's host rule guards against, and it is kept visible here, not hidden. **A Gitea executor in flight** (d3 T6c, and B3 seen from the executor's side). Step G's Gitea part goes in this order: 1. Record `grant.revoked {granted: false}` for each open Gitea decision (3c). 2. Await this agent's tracked Gitea executors and remote undos, if any are running. Each one ends `revoked` at its next `#giteaRevokedSince` check, or it committed `applied` before step 1. 3. Read the `gitea_grants` row again and run the steps above. `#giteaCloseRevoke` with `fire: true` lists the user's tokens and deletes every one. The list comes after step 2, so it includes any token an in-flight executor created. `#giteaRuns` is a single Set today. It becomes keyed by (business, agent) so step G can await one agent's executors alone. ### 3c. Open decisions A host named only by an open decision has no record. Step 3a still runs `remove` (a no-op) and records `grant.revoked {granted: false}`, as hosts.revoke does for a revoke with no record (N4). The executor's `#revokedSince` then ends that decision `revoked`. The Gitea side is the same through `#giteaRevokedSince`. This is the case where a decision is approved before the fire and executed after it (sage-2574). Both checks look for a `grant.revoked` of the agent recorded after the decision was raised (broker.mjs:1156 and :1678 at 47a35bf43). A fire's `grant.revoked` is always later than the approval, so the executor ends the decision whenever it runs. The fired agent keeps its roles entry, so `#hostDefinition` and `#giteaRole` still pass, and this check is the only thing that stops the decision. T6 and T6b test it. A delete also removes the roles entry, so the executor refuses there as well. The same holds for a decision raised but not yet approved (B1). It's in the set, so it gets its `grant.revoked {granted: false}`. If the human approves it later, its executor ends `revoked` (T6d). The decision stays approvable in the inbox. Approving it does nothing but end it, which `grant.failed` reports as it does for any revoked grant. ### 3d. A raise for a fired agent (d4, B1) `#hostGrantBinding` and `#giteaGrantBinding` both run inside the raise transaction (broker.mjs:1539 and :1548; the host raise's keyscan is awaited before that transaction). Both gain one check. The raise refuses `instance-fired` if the agent is: - firing (`#firing`); - fired, by the writer's own rule (instance-settings.mjs, hire): a roles entry, not in `launch.instances`, and not `launch.by`. A rehire lists the name again, so a rehired agent can be granted again. Because the check is in the transaction that records the decision, there's no gap. A raise that commits before step A's transaction is in the set. One that commits after it refuses. A raise for an agent whose delete's step G is running refuses `grant-revoke-running` (3e). No executor recheck is added. A raise can't commit after `fire.stopped`, and every decision raised before it is in the set and gets its `grant.revoked`. ### 3e. One step G per agent (d4, B3) The broker keeps `#revoking`, a map from (business, agent) to the verb whose step G is running. The verb sets it synchronously, in the same turn as the set read, before it answers. The job clears it in a `finally`. While it's set: | Caller | Answer | |---|---| | `instance.fire` | refuses `fire-in-progress` | | `lifecycleDelete` | refuses `grants-revoking`, no second job | | `hosts.revoke` / `grants.revoke` for the agent | refuse `grant-revoke-running` | | grant raise for the agent | refuses `grant-revoke-running` | | `grants.sweep` | skips this agent's entries and names it as `running` in its result | Two fires started together (the chief retrying after a client timeout): the first takes the lock before it answers, and the second refuses. The map is in memory. After a broker restart the job is gone and so is the lock. The instance is still firing, or the delete still has a non-empty set, and the repeat starts a fresh step G (section 5). A manual revoke can't race step G, since it refuses while the lock is held. Step G's transactions still check what they read: a `host_grants` row already gone at 3a step 3 records `grant.revoked {granted: false}` and goes on, as hosts.revoke does today for a revoke with no record. ## 4. A refusal (D4) Step G stops at the first grant that can't be revoked. Grants already revoked in this step G stay revoked and recorded, and the rest stay as they were. The job doesn't run step B: the organization file is unchanged, and the credentials are unrevoked, since `revoke()` never ran. A fire stays "firing". A delete's request was already closed with `grants-revoking` (section 2). The verb has already answered, so the refusal is a record, not a thrown code: - One grant-evidence event, `grant.revoke.refused {job, agent, grant: 'host:' | 'gitea', reason, verb: 'instance.fire' | 'lifecycle.delete', by}`. - `reason` is one of `host-unknown`, `host-remove-failed`, `gitea-admin-unavailable` and `gitea-left:`. - `grant-revoke-failed` is the code in the event and in the inbox message. A message lands in the human's inbox, as `grant.failed` does today (broker.mjs:1139). It names the instance, the grant and the reason, with the manual way out beside the retry (N2): > Fire of stopped: its grant on couldn't be revoked (). Run the fire again once > is reachable. If won't come back, run `mosaic hosts revoke ` first: the grant record > goes, the line is named as left, and the next fire completes and names that line (section 5). The delete's message has the same shape, with "raise the delete again". For Gitea, the way out is `mosaic grants sweep gitea` once Gitea is back. A job that throws something unexpected inside step G records `grant.revoke.refused` with reason `internal`, sends the same message, and releases the lock. There's no unhandled rejection. A failure at step B or C is `fire.failed` (section 2). **A restart mid-job (d6, N4).** The lock and the job are in memory, so a restart ends them with no end record. A fire would stay "firing" until the chief happened to repeat it. A delete's request is already closed, so no one would hear anything. Boot closes that gap: - A job is a start record (`fire.revoking` or `grants.revoking`) whose `job` has no end record. The end records are `fire.done`, `fire.failed`, `grants.revoked.done`, `grant.revoke.refused` and the new `grants.job.interrupted`. - When the broker starts, for each such job of each business, it records `grants.job.interrupted {job, agent, verb}` and sends the human one inbox message: > of : job was interrupted by a bus restart. Some grants may be revoked. Run the > fire again, or raise the delete again. The repeat goes on from what's left. - Recording the end record in the same transaction as the message means a second boot finds nothing to send. - Boot reads only events, with no network call, so it adds nothing to the start time beyond one query. ## 5. A repeat fire (D5) A repeat runs the whole fire again: the checks, step A (nothing left to stop), the set read, then step G, B and C in the job. A repeat while the job runs refuses `fire-in-progress` (3e). A repeat delete works the same way through the human's new request. - A host grant already revoked has no row and no open decision, so it isn't in the set. It's done. - A host grant that refused still has its row, so it's retried from 3a step 1. - Gitea (d4, N1): if the agent has entries in `#giteaLeft` or `#giteaOpenRevokes`, step G retries only those. It calls `#giteaCloseRevoke` for each, as `#sweepGitea` does per entry. It doesn't run the business-wide sweep, which would retry every agent's entries and settle every open rotation under the chief's name. Any entry still left for this agent refuses `gitea-left`. An entry left by an earlier manual `mosaic grants revoke` of the same agent blocks the fire the same way. That's intended: no grant outlives its instance, whichever verb started its revoke. These entries count toward a non-empty set (section 2, fire step 3), so they go to the job and never run inside the verb. - A host line left by an earlier manual `hosts.revoke` doesn't block the fire (Q1 below). These are the agent's `grant.revoke.closed` records with `hostLineLeft` and no later line removal for that tag. For each one whose host record still exists, step G sends `remove ` once. On success it records `grant.revoke.closed {revoke: , hostLine: 'removed', fire: true}`, so a later fire doesn't find it again. On failure it records nothing new and the fire goes on. The host and the tag go in `grantsLeft` (section 6). These leftover lines also make the set non-empty, so the retry runs in the job, inside B2's answer (dewey-0823 accepted the single retry on that condition). ## Rulings - **Q1 (sage-2573):** a leftover line doesn't block the fire. If the host is reachable, the fire tries the removal once, because a fired agent's key line still on a host is the exact risk this row closes. If that try fails, the fire doesn't refuse. It names the host and the leftover line in the result and in the event the human sees. The human accepted that residue when their manual `hosts.revoke` left it, so it shouldn't stop a fire forever on a dead host. Dewey may push back on the retry if it widens step G more than it's worth. - **Q2 (sage-2573):** a delete refused at step G closes its request through the launcher's normal refusal path, and the human raises it again. A request left taken would strand, which is the shape DEFERRED already records for the >64-stops done. Conditions: the refusal names the instance, the grant and the reason (the `grant.revoke.refused` event, section 4). The repeat delete skips grants already revoked, as the repeat fire does. T3b tests both. - **Q2 extended (sage-2575):** every delete of a granted instance takes two requests. The first starts the revoke job and refuses `grants-revoking`, and the human raises the delete again when the inbox says it's done. This keeps the cli and the host channel out of the row, and such a delete is rare. Conditions, each with a test: 1. The first refusal tells the human what happened and what's next: the named grants, and to raise the delete again when the inbox says done. A bare code isn't enough (T3). 2. A delete raised while the job runs refuses `grants-revoking` again and starts no second job (T3, T9). 3. The job's end message names each grant revoked. If one failed, it names the grant, the reason and the manual way out (`mosaic hosts revoke` or `mosaic grants revoke`). The next delete then refuses with that reason and doesn't complete over a live grant (T3b). 4. A delete of an instance with no grant completes in one request, exactly as today (T3c). 5. Section 2 says the fire uses the same answer-at-once job, and the fire's result and event name the job (T2). ## 6. The result and the event (D6) - **A fire with a non-empty set** answers at once with `job`, `revoking` and `done: false`, and records `fire.revoking {job, grants}` (section 2). - **The job's outcome** is recorded, not returned. `fire.done` (step C) carries `job` and `grants: [{grant: 'host:' | 'gitea', revoke, granted}]`, plus `grantsLeft` for any leftover from section 5's last bullet. So the event the human sees names them. - **A delete's job** records `grants.revoking {job, ...}` at the start and `grants.revoked.done {job, agent, verb: 'lifecycle.delete', grants, grantsLeft}` at the end. It sends the inbox messages in section 2. - **A fire with an empty set** carries none of these keys, so its shape is exactly today's. ## 7. What doesn't change (brief item 4) - `hosts.revoke`, `grants.revoke` and `grants.sweep`: same callers and the same records. Their refusals stay the same (`tester-dev-only` included) with one addition from section 3e (d6, N5): while a step G job holds the agent's lock, `hosts.revoke` and `grants.revoke` for that agent refuse `grant-revoke-running`, and `grants.sweep` skips the agent and names it as `running`. hosts.revoke's own order stays local first. - No new verb, no Console route, no CLI change, no schema change. Every new record is an `events` row in the existing grant-evidence shape. - `revoke()` in runtime.mjs and the organization writer don't change. - `server.mjs`, `process.mjs`, `cli/src/host.mjs` and `cli/src/launcher.mjs` don't change. Both verbs stay synchronous. The launcher already closes a request with whatever code `lifecycleDelete` refuses. - New codes: `fire-in-progress`, `grants-revoking`, `grant-revoke-running`, `instance-fired`, and `grant-revoke-failed` in events and messages. New event types: `fire.revoking`, `grants.revoking`, `grants.revoked.done`, `grant.revoke.refused`, `fire.failed` and `grants.job.interrupted`. - The broker's start gains one events query for interrupted jobs (section 4, N4). `lifecycleDone` checks a refusal against `LIFECYCLE_CODE = /^[a-z-]{1,64}$/` (broker.mjs:149), and `grants-revoking` matches it. ## 8. Tests (each in the owning package, whole files) The bus tests use the fake installer and the fake Gitea from rows 235 and 236. They wait on the job with a new trusted test API, `grantsIdle()`, which resolves once every step G job and Gitea executor has ended. It's the same shape as `giteaIdle()`. Unless a test says otherwise, every fire or delete below is awaited that way. - T1 (pins section 1 before the change): a fire of an instance with a host grant refuses `credential-revoke-failed` today. A delete leaves the `host_grants` row. Then: what the next start does with a record whose key file is gone. - T2: a fire with a host grant on two hosts and a Gitea grant. The verb answers with a `job` id, `revoking` with all three, and `done: false`, and `fire.revoking` carries the same job (sage-2575, condition 5). After the job: - all three are revoked, each with `grant.revoked` and `grant.revoke.closed` as the manual verbs record them, plus `fire: true`; - the line is gone on both fake hosts, every token is deleted, and login is prohibited at the fake Gitea; - `fire.done` names all three; - that agent's `host.run` and Gitea use both refuse. - T3 (sage-2575, conditions 1-3): the same grants through the delete (`lifecycleTake` then `lifecycleDelete`). - The first delete refuses `grants-revoking` and starts the job. The human's inbox gets the start message naming the job and all three grants, and saying to raise the delete again when it's done. - A delete raised while the job is held refuses `grants-revoking`, starts no second job and sends no second message. - After the job, the end message names the job and each grant revoked. - A third request completes as today. - T3b (Q2, sage-2575 condition 3): a delete whose job refuses (host B unreachable). - The end message and `grant.revoke.refused` name the instance, `host:B`, the reason and `mosaic hosts revoke`. - The file and the instance are unchanged. - The next delete refuses `grant-revoke-failed`, not `grants-revoking`. Its message names the earlier job's reason and the new job, and the instance is not deleted. - With B back, that retry job revokes only B and Gitea, skipping host A, and the delete after it completes. - A second case clears B by hand with `hosts.revoke` before the next delete. That delete starts a job only for Gitea and the leftover line. - T3c (sage-2575 condition 4): a delete of an instance with no grant completes in one request. Same result, same events counted, no inbox message. - T4: host B unreachable in a fire. The job records `grant.revoke.refused` naming the agent, `host:B` and `host-remove-failed`, and the inbox message names `mosaic hosts revoke` (N2). Afterwards: - the file is unchanged and the instance is still firing; - host A is revoked and recorded, and host B's row and key file are still there; - the Gitea grant is untouched, since it comes after B in the order; - the credentials are unrevoked. A repeat with B reachable completes and revokes only B and Gitea. - T4b (Q1): an earlier manual `hosts.revoke` left a line on host C. A fire with C reachable removes it and records the close. With C unreachable, the fire completes and names C and the tag in `grantsLeft` and in `fire.done`. - T5: Gitea's admin token can't be checked. The job refuses with the Gitea grant intact. - T5b: a token DELETE fails. The job refuses `gitea-left`. A repeat with Gitea back retries only this agent's entries and completes. Another agent's left entry is still left afterwards (N1). - T6 (sage-2574): a host grant decision approved before the fire, executed after it. The fire records `grant.revoked {granted: false}` for (agent, host). The executor runs after `fire.done` and ends the decision `revoked`: no `host_grants` row, no key file, no `add` sent to the host. - T6b (sage-2574): the same for a Gitea grant decision. The executor ends it `revoked` before any Gitea call: no `gitea_grants` row, no token file, and the fake Gitea saw no token create. - T6c: a host or Gitea executor already in flight when step G starts. - The host side aborts its claim (3a step 1) and ends `revoked`. - The Gitea side: step G awaits the executor (3b), and then the fake Gitea holds no token for the user. That includes one the executor created mid-flight. - T6d (B1): a host decision and a Gitea decision, raised before the fire and still undecided. The fire puts both in the set. The human approves both after `fire.done`, and both executors end `revoked`, with nothing applied. - T6e (B1): a host raise and a Gitea raise refuse `instance-fired`: - while the instance is firing; - after `fire.done`. After a rehire of the same name, a raise goes through. While a delete's job runs, a raise refuses `grant-revoke-running`. - T7: an instance with no grant fires exactly as today, synchronously. Same result keys, no `revoking`, and the same events, counted. - T8: hosts.revoke and grants.revoke keep their records, their order and their tester refusals. - T9 (B3): with the job held on a remove: - a second fire refuses `fire-in-progress`; - a delete of the same agent refuses `grants-revoking` and starts no second job; - `hosts.revoke` and `grants.revoke` for the agent refuse `grant-revoke-running`; - `grants.sweep` names the agent as `running`. After a release, one set of `grant.revoked` records exists per grant. - T10 (B2): the fake installer holds a remove and the fake Gitea holds a DELETE, neither answering until released. - `instance.fire` and `lifecycleDelete` both answer before either is released. The test checks this with a timer, not a wall-clock bound. - Meanwhile the broker answers another verb from another session. - After release, the jobs complete. - A cli test drives the launcher's delete pass against a bus whose remove is held. The pass closes the request `grants-revoking` and the host channel stays up. - T11 (d6, C1): a fire whose job is held on a remove. While it's held, the chief's run ends and its claim is released, and processing is halted. After the release, the job completes: `fire.done` is recorded with the job, the consumed `action.allowed` event is there, a second use of the same authority refuses `decision-consumed`, and the instance is out of the file. No `fire.failed` and no `grant.revoke.refused`. - T12 (d6, C1): a fire with a grant whose step B refuses in the job. The test uses T1's setup, where `revoke()` refuses `credential-revoke-failed`. After the job: - `fire.failed` carries the job, step `B` and `credential-revoke-failed`, and the inbox message names the job, the revoked grants, the step and the code; - there's no `grant.revoke.refused` and no `fire.done`; - the instance is still firing and still in the file. With the cause cleared, a repeat fire finds an empty set, runs B and C synchronously and completes. - T13 (d6, N4): a fire and a delete, each with its job held, then the broker is closed and a new one opened on the same data root. The new broker records `grants.job.interrupted` for both jobs and sends two inbox messages, one saying to fire again and one to raise the delete again. A third broker on the same root sends nothing more. A job that ended before the restart gets no interrupted record. Files (N3). Every file listed runs whole in the manifest round, touched or not: - New: `packages/bus/tests/fire-revokes-grants.test.mjs` holds T2-T13. - Changed: - `grant-host.test.mjs` and `gitea-grant.test.mjs`: T1's pins, and the T8 regressions next to their existing cases; - `packages/cli/tests/lifecycle.test.mjs`: the T10 launcher pass; - `fake-gitea-api.mjs` and `host-fixtures.mjs`, if T10's holds need a hook there. - Run whole because they call `instance.fire` or `lifecycleDelete` at 47a35bf43: - bus: `chief-of-staff`, `lifecycle-revoke`, `lifecycle-verbs`, `reload`; - cli: `lifecycle`, `instances`; - harness: `tools`. - Also run: `grant-executor.test.mjs` (the executors' `#giteaRuns` change) and `gitea-grant-docker.test.mjs` (brief gate). - `server.mjs`, `process.mjs`, `host.mjs` and `launcher.mjs` are unchanged, so no test changes there for this row. ## 9. Mutants - M1: step G skips the host revokes. Killed by T2. - M2: step G skips the Gitea revoke. Killed by T2. - M3: a refused grant revoke is ignored and the job goes on to step B. Killed by T4. - M4: `lifecycleDelete` skips step G. Killed by T3. - M5: the fire's host revoke goes local first (`lineFirst: false`). Killed by T4, because B's row is gone. - M6: a Gitea revoke with something in `serviceLeft` counts as done. Killed by T5b. - M7: open decisions left out of the set. Killed by T6 and T6b. - M8: step G skips the leftover-line retry. Killed by T4b. - M9: a failed leftover-line retry refuses the fire. Killed by T4b. - M10: the set takes only approved decisions (d3's rule). Killed by T6d. - M11: the raise bindings skip the fired check. Killed by T6e. - M12: no `#revoking` lock. Killed by T9. - M13: the verb awaits the job before it answers. Killed by T10. - M14: step G's Gitea part doesn't await the in-flight executor. Killed by T6c. - M15: step G's Gitea retry runs the business-wide `#sweepGitea`. Killed by T5b. - M16: the delete's first refusal sends no start message. Killed by T3. - M17: a delete after a refused job refuses `grants-revoking` instead of `grant-revoke-failed`. Killed by T3b. - M18: a delete of an instance with no grant goes through the job. Killed by T3c. - M19: the job's step C uses `#consumeAuthority`, with the holder and processing checks. Killed by T11. - M20: a step B or C failure in the job is recorded as `grant.revoke.refused internal`. Killed by T12. - M21: boot skips the interrupted-job check. Killed by T13. ## 10. Order and rebase Built from 47a35bf43 for this read. The code goes on whatever refactor is when the build starts. Rows 274, 256 and 194's follow-ups touch the broker and instance-settings, so expect a rebase and a whole rerun. The row lands before the .53 Case 1 install hires an instance with a grant. It doesn't hold the dragon-lin re-pin. ## Build notes Dewey approved d6 (dewey-0829, 10:11Z 10-22) with two build-time items. Both are test details, so there's no new draft: - N7, T12's setup. T1's setup makes `revoke()` refuse because the grant's `host.` file is still in the store. Step G's `hostKeys.revoke` removes that file before step B, so it can't be T12's cause. T12 uses a step B refusal that survives step G, for example a stray file in the agent's store directory or a roles-entry credential whose revoke fails. The test asserts it reaches `fire.failed` with step `B`. - N8, `decision-consumed` in T11. A chief's fire is within-role: `#checkAuthority` returns `{ class }` with no decision, and the lifecycle verbs take none (broker.mjs:3712). `#spent` never applies to a fire. T11 drops the second-use `decision-consumed` assertion. Instead, a repeat fire after `fire.done` takes the delisted path and answers done. Section 2's `decision-consumed` example for `fire.failed` can't occur, so the build uses only codes that can. Choices made in the code (b1, broker.mjs only so far). None changes an approved section; each fills in a detail the design left open. Dewey, flag any you read differently. - Record writers. `writeGrantEvent` takes only `grant.*` actions. So `fire.revoking`, `fire.done`, `fire.failed` and `grants.job.interrupted` are not grant evidence. `fire.*` goes through `#event` under the chief's session (the job keeps a copy, `sc`). `grants.revoking`, `grants.revoked.done` and `grants.job.interrupted` go under the bus session `{ role: null, run: null }`, subject the agent. `grant.revoke.refused` is grant evidence, as section 4 has it. - A fired Gitea user. The `gitea_users` row survives a fire (#436b section 6), so on its own the row would never let a fired agent's set empty. The user counts as pending until a later Gitea `grant.revoked` with `fire: true` and a non-null `user` names the agent, and counts again after a new grant step (a rehire). - Open Gitea decisions (3c) end with a `grant.revoked` that has `user: null`, `tokens: []`, `fire: false` and `decisions: [ids]`, plus an immediate close with `serviceLeft: []`. `#giteaRevokedSince` then ends them. Since `user` is null, the pair never counts as the user's fire. - Leftover lines (section 5, Q1). A line is in the set as `line:` when its tag is in `#hostLinesLeft`, its host isn't one the agent holds a grant on, and no `fire.done` or `grants.revoked.done` for the agent since the line was last named left lists it in `grantsLeft`. A line removed in step G gets `grant.revoke.closed` with the old revoke id, `hostLine: 'removed'` and the flag. `#hostLinesLeft` now clears a tag on a later close with `hostLine: 'removed'` and `fire` or `delete` set, so the line also leaves `hosts sweep`'s list. The removed lines appear in `grants` as `{ grant: 'line:', revoke, tag }`. - 3a's wait for host executors. `#runHostGrant` marks itself in `#hostRuns` from its claim to its release. Step G's host part runs `abortClaim`, waits for every mark on (agent, host), and sends the remove. If a new mark appeared during the remove, it runs again, up to five rounds, and then refuses `host-remove-failed`. From the last check to the step 1 commit there's no await. - `hosts sweep gitea` (3e) skips the open revokes, settles and left entries of every agent a job holds, and keeps those entries in `left`. Its answer has `running: [agents]` only when there's at least one. The retry of one left entry is now `#giteaRetry`, which the sweep and step G share. The behavior is unchanged. - Step G's Gitea retries record a sweep-shaped pair for the agent: `grant.sweep { agent, job }`, then the closes and retries, then `grant.swept` whose `left` keeps every other agent's entries. The pair is written first, so a serviceLeft from a close between the two can't drop out of `#giteaLeft`. - `#giteaRuns` is now a Map from each promise to its `business/agent` key, or null for business-wide work. Step G waits for its own agent's keys and the null ones. `#track` adds `.catch` to its `finally` chain, so a rejected executor can't leave an unhandled rejection behind. - A delete repeat after a refusal. `grant-revoke-failed` is returned only when the agent's last delete job end was a `grant.revoke.refused` whose grant is still in the set. If that grant was cleared by hand and only a leftover line remains, the repeat answers `grants-revoking`. - The delete job's reason, in `grant.revoked`, is `lifecycle.delete `. - Step G's messages go to the human's inbox alone (no decision, so no raising role to thread to), through the same insert and route as `#grantNotice`. - `#recordAuthority` is `#consumeAuthority` less `#checkAuthority`'s holder and processing checks, for a fire's step C (C1). Choices made in the tests (b2). Each changes where a test sits or how it checks, not what it checks: - T1 is in `lifecycle-revoke.test.mjs`, not `grant-host.test.mjs` and `gitea-grant.test.mjs`. T1 pins the runtime's `revoke()` (step B's credential check) and the start sweep, and only `lifecycle-revoke` builds a broker through `startBroker` with the real credential store. Its `setup` takes a new `seed` hook that writes a host record and grant before the start. - T8 is in `grant-executor.test.mjs` for `hosts.revoke`, where G12, G13, G26 and G27 already are, and in `gitea-grant.test.mjs` for `grants.revoke`. G12 already pins the host verb's two records, their bodies and their order, and G26 its role refusals, so T8 adds what no test had: `tester-dev-only` once `bus.tester` is off, and that a verb revoke writes no `fire.*` or `grants.*` record. - T2-T13 use a fake installer and a fake organization in `fire-revokes-grants.test.mjs`. The organization's `instance.fire` delists and reloads the business, as the writer does, and can throw a set error, which is how T12 gets its step B cause (N7): a `credential-revoke-failed` that survives step G. - "A fired agent's `host.run` and Gitea use refuse" is checked through the state those uses read: no `host_grants` row, no key file, a revoked credential, and no Gitea token for the user. - T10's cli case runs the real runtime and launcher. The seeded grant is on a TEST-NET host (192.0.2.10), so the job's remove fails or waits. The first request closes `grants-revoking` either way, and the case waits on `grantsIdle()` before close. - The mutants are `scratch/megavolt-r281/r281-mutants.sh`, one sed each on `broker.mjs`, run against `fire-revokes-grants.test.mjs`. M13's verbs are synchronous, so its mutant makes the fire return the job's promise, and the call waits for the job. That's what the verb awaiting its job would look like. Rebase onto 46dedf925, after row 256 (sage-2597). It's on branch `megavolt/row281-b`, and `megavolt/row281` stays where it is. `git range-diff 0165be266..bd0bb96ae origin/refactor..HEAD` gives `=` for all nine commits, and no path changes on both sides. Row 256 threads the actor through `planDelete`, `checkDelete` and `instances.check`, and adds `protectedRefusal`. That doesn't change what a delete calls in this row: - At the verb, `instances.set` op delete already ran `instances.check` with `s.human` before 256. A creative instance's delete by anyone but the human now refuses there, before any request exists, so no step G job starts for it. - In the pass, step G runs in `lifecycleDelete` before `#organization('instances.set')`, and the actor is still `r.by`. `planDelete` there can still refuse after step G has revoked. Before 256 a references or last-listed refusal could do the same thing, so the order is unchanged. A delete the verb accepted only meets a new refusal in the pass if the file changed between the two. - `read(business, actor)` changes only the listing's `endProtected`. The broker and these tests don't read it. The T10 cli case deletes `sec-1` (security-reviewer), which isn't END_PROTECTED. Rebase onto 980cec88c, after row 277 (sage-2605). It's on branch `megavolt/row281-c`. `git range-diff 46dedf925..7454be493 980cec88c..c174f37ac` gives `=` for all ten commits. Row 277's `thread.claims` change to `broker.mjs` doesn't touch step G or any line the mutants change: `scratch/megavolt-r281/anchors.sh` counts each M1-M21 sed anchor in the rebased `broker.mjs`, and all 28 counts are 1. The mutants script runs it after a green base and stops if a count isn't 1. b3, after the first base run on c174f37ac (11:24:47Z) came back red. These tests had never run before: b2 was written under git and docs only. bus: 120 tests, 116 pass, 4 fail. cli lifecycle: 24 tests, 23 pass, 1 fail. Every older test in those files passed. No mutant ran. Each failure is a test fault, and b3 changes no code: - T4b, `fire-revokes-grants`: **test changed**. The test read `manual.revoke`, but the `hosts.revoke` answer has never carried `revoke`: `#revokeHost` drops it, so the verb's answer keeps its pre-281 shape. The fire's close names the manual revoke, as section 3a says. The test now takes that id from the manual's `grant.revoked` event. - T5b, `fire-revokes-grants`: **test changed**. The job refused `gitea-left:9,tokens`, and the test expected `gitea-left:`. Section 4 sets the reason as `gitea-left:`, the ids left at Gitea. For a fire, a close's `serviceLeft` is the N10 list from before 281: each token id whose DELETE failed, then `tokens` or `login` for a fire step that failed (`#giteaCloseRevoke`). With the DELETE failing, `#giteaFire`'s token list never clears, so `tokens` is a real entry left at Gitea. It belongs in the reason. The test now expects `gitea-left:,tokens`. - T6b, `fire-revokes-grants`: **test changed**. After the fire, the executor made 4 Gitea calls where the test expected none. They are step 0's admin checks C1 to C4: version, self, token scopes and the repo check, all GET. Step 1's `#giteaRevokedSince` then ends the decision `revoked` before any write. Section 3c of d6 names that step 1 check as the only thing that stops the decision. T6b's line in section 8 says "before any Gitea call", but the evidence it lists is no row, no token file and no token create. That phrase was wrong, because step 0 always runs first. The approved text stays as it is, and this note is the correction. The test now checks that every call after the fire is a GET and that no token create happened. Moving the revoked check ahead of step 0 would change the executor for every revoke, not just step G's, and the design doesn't ask for that. - T8, `gitea-grant`: **test changed**. The human revoke's events between `grant.revoked` and `grant.revoke.closed` include its 5 `grant.call`/`grant.result` pairs (`#giteaLog` logs both around every Gitea call). T8 checks that a human revoke writes no job records, not that it makes no calls. It now leaves the call log out before comparing. - T10, cli `lifecycle`: **test changed** (sage-2615 asked about this one). The `credential-file` came from `startBroker`, not from the delete and not from Gitea. The seeded host grant gives `sec-1` a `host.web1` store reference. `Credentials` checks the store root (`/secrets/agents`) with `privateDir` at start, and the fixture had no store root. That check is row 199's, and it is right to fail closed. The test now makes the store root at 0700 under `dirname(p.f.config)`, as the bus host fixtures do, never under the real config. sage-2575's condition still holds. This business has no Gitea grant, so `#grantSet` adds no `gitea` entry (`#giteaPending` and `#giteaUnfired` read only the store), and step G never reaches `#stepGGitea`. A delete with no Gitea grant reads no Gitea credential file. ### b4a: the M1 hang's cause (mos-1273, sage-2636) The first M1-M21 run on 06ee5ed6b hung on M1. All 22 tests ended in about 80 s (16 fail, 5 pass, 1 cancelled), but the file's process stayed alive until the harness killed it at 1762 s. This is a test fault, and no code changed. - Cause: T6c holds `web1:add`, and the fake `install` loops in `pause()` on `sleep(5)` while the key is held. Only the fire's abort signal can end that loop, and M1 removes the host revoke that sends the abort. In M1's log, T6c fails at its `until` on line 683, before it deletes the key on line 691. So the loop keeps a timer alive forever. That reading comes from the log and the file. The rerun checks it: before M1-M21 it runs M1 once without `--test-force-exit`, and that run must exit by itself inside 180 s. - Fix: setup's `t.after` clears `held` and `failing` before it closes the store, so any held loop ends with its test. The base is unchanged because every passing test already releases its keys. - Sage's second candidate, the Gitea `gate` in T6c and T10, isn't a live handle. The fake server's `t.after` closes all its connections, and a parked promise with no timer doesn't keep node alive. Both tests now also release the gate in `t.after` (`go()` only, not `off()`: a second `off()` would splice index -1 and remove another rule). - The runner's 180 s timeout with `--test-force-exit` (sage-2633) still caps any future hang. ### b4: T6b pins the four reads (dewey-0839, sage-2627) - T6b: **test changed**, stricter. `slice(calls).every(GET)` also passed with zero calls. It now deep-equals the exact list in order: `GET /api/v1/version`, `GET /api/v1/user`, `GET /api/v1/token`, `GET /api/v1/user/repos?limit=1&page=1` (step 0's C1 to C4, `gitea-grant.mjs` adminChecks). It also fails if step 1 stops ending the run and step 2's reads go out. The title says "before any Gitea write". ### T14 and M22: a declined decision on a removed host (sage-2653, dewey-0844, N1) Test only, written before the slot. Not run yet. - T14 (new, in `fire-revokes-grants.test.mjs`): coder's web1 decision is raised and left undecided. The human then runs `hosts.remove web1`, which goes through because hosts.remove doesn't check open decisions (N1). The fire starts a job whose set is `host:web1`. Step G refuses `host-unknown` with one `grant.revoke.refused`, and one inbox message names the job, web1 and host-unknown. There's no fire.done. The human resolves the decision `no`. No executor runs (`#approvedGrant` is null for a no), so the closed check in `#openAgentDecisions` is the only thing that takes it out of the set. The repeat fire answers with no job and records one fire.done with no job. - M22 deletes that closed check (`broker.mjs:1684`). The declined decision stays open, so the repeat fire starts a second job and refuses host-unknown again, and T14 fails on `repeat.job`. M10 mutates the same line in another way (approved-only), and T6d kills it. M22 runs on the fire-revokes file like M1-M21, and its anchor makes 29. - N1's hint text (hosts.remove naming the open decision) is a source change. It stays out of this row and goes to a DEFERRED line that Sage files at landing. - If T14 is red on its first run, it goes to Sage and then Mos untouched, because 281 has had its one placed red (mos-1272). ### Gate plan (sage-2649, dewey-0840, mos-1275) Written before the slot. The slot runs on row281-e after the rebase onto 278's landing, and every step stops at the first red. 1. The base: the touched files whole (fire-revokes-grants, lifecycle-revoke, grant-executor, gitea-grant in bus; lifecycle in cli). Then the 29 anchors, the M1 exit check without `--test-force-exit`, and M1-M22 under `timeout -k 10 180` with `--test-force-exit`. 2. dewey-0840's files whole: bus chief-of-staff, lifecycle-verbs, reload, grant-host and gitea-grant-docker; cli instances; harness tools. If gitea-grant-docker skips, the report names the skip and its reason. 3. bus, business and cli whole. 4. Every `scripts/test-*.sh`, each as `timeout -k 10 LIMIT bash scripts/test-X.sh`, and no script is edited. LIMIT is Gander's row 278 limit (`agents/gander/work/queue-278/SHELL-LIMITS.json`, sage-2646): 3x the longest successful run from Gander's row 196 receipts, plus 120 s headroom. I checked it against two more records. One is my row 236 gate at bd5f11ded (10-22). The other is 278's run 3 (`run3/whole/results.json`). Neither has a longer run, so every limit is at least 3x the longest of all three. A script with no limit stops the gate before it starts. Any timeout goes to Sage, then Mos, unplaced. | Script | Row 236 | 196 receipts | 278 run 3 | Longest | 3x | LIMIT | |---|---|---|---|---|---|---| | test-auth | 1 s | 1.9 s | 0.8 s | 1.9 s | 6 s | 126 s | | test-conductor | 6 s | 8.1 s | 6.1 s | 8.1 s | 25 s | 145 s | | test-config | 1 s | 3.2 s | 1.9 s | 3.2 s | 10 s | 130 s | | test-discord | 8 s | 8.8 s | 7.4 s | 8.8 s | 27 s | 147 s | | test-extension-package | 1 s | 3.0 s | 1.4 s | 3.0 s | 10 s | 130 s | | test-foundation | 39 s | 39.7 s | 39.9 s | 39.9 s | 120 s | 240 s | | test-land-work | 11 s | 12.6 s | 11.3 s | 12.6 s | 38 s | 158 s | | test-queue | 20 s | 23.4 s | 21.2 s | 23.4 s | 71 s | 191 s | | test-release | 2 s | 12.2 s | 2.7 s | 12.2 s | 37 s | 157 s | | test-task | 14 s | 36.8 s | 13.8 s | 36.8 s | 111 s | 231 s | ## Runs The manifest-round slot on row281-e 7979fb41e (base 385a7194c, row 278's landing), CLEAR sage-2661. Runner `r281-slot.sh` (scratch), from `scratch/megavolt-r222`: 12:43:17Z to 12:59:25Z, 10-22, SLOT_EXIT 0. No red, no timeout. Every step ran in a detached worktree at the candidate, with TMPDIR under `/mnt/storage/scratch/tmp`. The gate worktree was clean at the end, and the trap removed both worktrees. 1. The base, each file whole, plain `node --test`: bus fire-revokes-grants, lifecycle-revoke, grant-executor and gitea-grant 121/121, 0 skipped, 22 s; cli lifecycle 24/24, 0 skipped, 13 s. T14 passed on its first run. All 29 anchors matched once. The M1 exit check exited by itself: rc 1, 17 fails (the 16 on 3f29375a8, plus T14), 63 s. M1-M22: 22 killed, 0 survived (Mutants). 2. dewey-0840's files whole: bus chief-of-staff, lifecycle-verbs, reload, grant-host and gitea-grant-docker 76/76, 0 skipped, 11 s (gitea-grant-docker ran, with no skip); cli instances 14/14, 0 skipped, 3 s; harness tools 23/23, 0 skipped. 3. Whole packages: bus 819/819, 23 s; business 231/231, 3 s; cli 837/837, 591 s. Each has 0 skipped. 4. The shells, each under `timeout -k 10 LIMIT`, all rc 0: | Script | Result | Time | LIMIT | |---|---|---|---| | test-auth | 15 passed, 0 failed | 1 s | 126 s | | test-conductor | 17 passed, 0 failed | 6 s | 145 s | | test-config | 31 passed, 0 failed | 1 s | 130 s | | test-discord | 81 passed, 0 failed | 8 s | 147 s | | test-extension-package | 18 passed, 0 failed | 1 s | 130 s | | test-foundation | 44 passed, 0 failed | 36 s | 240 s | | test-land-work | 86 passed, 0 failed | 10 s | 158 s | | test-queue | node 156/156; 28 passed, 0 failed | 20 s | 191 s | | test-release | 14 passed, 0 failed | 3 s | 157 s | | test-task | 91 passed, 0 failed | 13 s | 231 s | Skips, from a grep of every shell log for `^skip`, `# skip` and `SKIP` (dewey-0845): - test-task, lines 87 and 93: `skip live provider cases (row 244: no provider call in the gate)`. Row 244 keeps provider calls out of the gate by design. The log has no Docker lines and no other skip. - test-queue, line 34: `skip queue verify and render --check: this checkout (.../r281-gate-wt) is not the queue's canonical root (/mnt/storage/src/mosaic-stack)`. The gate runs in a worktree, so the canonical-root checks are skipped. - No other shell log, and no node suite above, has a skip. ## Mutants Run on b4a 3f29375a8 (base 6af85df4b), 12:16:18Z to 12:20:15Z, 10-22, slot sage-2643; runner `r281-mutants.sh` (scratch). Base first, plain `node --test`, each file whole: bus fire-revokes-grants, lifecycle-revoke, grant-executor and gitea-grant 120/120, 0 skipped; cli lifecycle 24/24, 0 skipped. The worktree was clean after the base, and all 28 sed anchors matched once in `broker.mjs`. The M1 exit check (mos-1273) applied M1 and ran fire-revokes-grants without `--test-force-exit`. It exited by itself, rc 1, with 16 fails, in 63 s. The first run on 06ee5ed6b hung there until it was killed at 1762 s (b4a above). Each mutant ran fire-revokes-grants.test.mjs whole under `timeout -k 10 180 node --test --test-force-exit` (sage-2633). Every one ran all 21 tests, with no load error and no timeout. | Mutant | Change | Verdict | rc | Fails of 21 | Time | Section 9 names | Failing tests (from the log) | |---|---|---|---|---|---|---|---| | M1 | skip-host-revokes | KILLED | 1 | 16 | 63 s | T2 | T2 T3 T3b×2 T4 T4b×2 T6 T6c T6d T6e T9 T10 T11 T12 T13 | | M2 | skip-gitea-revoke | KILLED | 1 | 12 | 12 s | T2 | T2 T3 T3b×2 T4 T5 T5b T6b T6c T6d T9 T10 | | M3 | refusal-goes-on | KILLED | 1 | 3 | 3 s | T4 | T4 T5 T5b | | M4 | delete-skips-step-g | KILLED | 1 | 6 | 3 s | T3 | T3 T3b×2 T6e T10 T13 | | M5 | host-local-first | KILLED | 1 | 4 | 3 s | T4 | T2 T3b×2 T4 | | M6 | service-left-done | KILLED | 1 | 1 | 3 s | T5b | T5b | | M7 | set-no-open-decisions | KILLED | 1 | 4 | 3 s | T6, T6b | T6 T6b T6c T6d | | M8 | skip-line-retry | KILLED | 1 | 3 | 4 s | T4b | T3b T4b×2 | | M9 | line-retry-refuses | KILLED | 1 | 2 | 3 s | T4b | T3b T4b | | M10 | approved-only | KILLED | 1 | 1 | 3 s | T6d | T6d | | M11 | raise-no-fired-check | KILLED | 1 | 3 | 3 s | T6e | T4 T6e T12 | | M12 | no-lock | KILLED | 1 | 3 | 3 s | T9 | T3 T6e T9 | | M13 | verb-awaits-job | KILLED | 1 | 14 | 3 s | T10 | T2 T4 T4b×2 T5 T5b T6 T6b T6c T6d T10 T11 T12 T13 | | M14 | no-executor-wait | KILLED | 1 | 1 | 3 s | T6c | T6c | | M15 | business-wide-sweep | KILLED | 1 | 1 | 4 s | T5b | T5b | | M16 | no-start-message | KILLED | 1 | 2 | 3 s | T3 | T3 T3b | | M17 | repeat-grants-revoking | KILLED | 1 | 1 | 3 s | T3b | T3b | | M18 | empty-set-job | KILLED | 1 | 4 | 4 s | T3c | T3 T3b×2 T3c | | M19 | step-c-consume | KILLED | 1 | 1 | 4 s | T11 | T11 | | M20 | bc-as-internal | KILLED | 1 | 2 | 3 s | T12 | T6e T12 | | M21 | no-interrupted-check | KILLED | 1 | 1 | 4 s | T13 | T13 | Every mutant fails the test section 9 names for it; none was killed only by a different test. "T3b×2" and "T4b×2" are the two tests that share each id. The failing-test ids are taken from each log's "failing tests" list (dewey-0841). 21 killed, 0 survived. b4 makes one T6b assertion stricter after this run, and fire-revokes-grants is rerun whole on b4 (Runs). ### On row281-e 7979fb41e (sage-2661) 12:43:17Z to 12:47:12Z, 10-22, the first part of the manifest-round slot (Runs). Same runner and flags as above, with M22 added (T14 and M22 in the build notes). Every mutant ran all 22 tests, with no load error and no timeout. | Mutant | Change | Verdict | rc | Fails of 22 | Time | Section 9 names | Failing tests (from the log) | |---|---|---|---|---|---|---|---| | M1 | skip-host-revokes | KILLED | 1 | 17 | 62 s | T2 | T2 T3 T3b×2 T4 T4b×2 T6 T6c T6d T6e T9 T10 T11 T12 T13 T14 | | M2 | skip-gitea-revoke | KILLED | 1 | 12 | 13 s | T2 | T2 T3 T3b×2 T4 T5 T5b T6b T6c T6d T9 T10 | | M3 | refusal-goes-on | KILLED | 1 | 4 | 3 s | T4 | T4 T5 T5b T14 | | M4 | delete-skips-step-g | KILLED | 1 | 6 | 3 s | T3 | T3 T3b×2 T6e T10 T13 | | M5 | host-local-first | KILLED | 1 | 4 | 3 s | T4 | T2 T3b×2 T4 | | M6 | service-left-done | KILLED | 1 | 1 | 3 s | T5b | T5b | | M7 | set-no-open-decisions | KILLED | 1 | 5 | 3 s | T6, T6b | T6 T6b T6c T6d T14 | | M8 | skip-line-retry | KILLED | 1 | 3 | 3 s | T4b | T3b T4b×2 | | M9 | line-retry-refuses | KILLED | 1 | 2 | 3 s | T4b | T3b T4b | | M10 | approved-only | KILLED | 1 | 2 | 3 s | T6d | T6d T14 | | M11 | raise-no-fired-check | KILLED | 1 | 3 | 3 s | T6e | T4 T6e T12 | | M12 | no-lock | KILLED | 1 | 3 | 3 s | T9 | T3 T6e T9 | | M13 | verb-awaits-job | KILLED | 1 | 15 | 3 s | T10 | T2 T4 T4b×2 T5 T5b T6 T6b T6c T6d T10 T11 T12 T13 T14 | | M14 | no-executor-wait | KILLED | 1 | 1 | 3 s | T6c | T6c | | M15 | business-wide-sweep | KILLED | 1 | 1 | 3 s | T5b | T5b | | M16 | no-start-message | KILLED | 1 | 2 | 3 s | T3 | T3 T3b | | M17 | repeat-grants-revoking | KILLED | 1 | 1 | 3 s | T3b | T3b | | M18 | empty-set-job | KILLED | 1 | 4 | 3 s | T3c | T3 T3b×2 T3c | | M19 | step-c-consume | KILLED | 1 | 1 | 3 s | T11 | T11 | | M20 | bc-as-internal | KILLED | 1 | 2 | 3 s | T12 | T6e T12 | | M21 | no-interrupted-check | KILLED | 1 | 1 | 3 s | T13 | T13 | | M22 | no-closed-check | KILLED | 1 | 1 | 3 s | T14 | T14 | 22 killed, 0 survived. Every mutant fails the test it's named for, and M22 fails T14 alone. Compared with 3f29375a8, the only new failures are T14's (under M1, M3, M7, M10, M13 and M22), and every other failing set is unchanged.