row 281 design d1: a fire or a delete revokes the instance's host and Gitea grants (sage-2568, #1786)

2026-10-22 04:54:22 -05:00
1 parent 47a35bf436
commit dfebd16cc1
@@ -0,0 +1,230 @@
 # Row 281 build notes: a fire or a delete revokes the instance's grants (#1786)
 Builder Megavolt, reviewer Dewey, gate owner Sage. Design draft d1 for Dewey's approval. No code until it is
 approved. The results sections are filled in after the runs on Sage's gate.
 ## Sources
 - Brief: `docs/plans/2026-10-22_fire-revokes-grants.md`, "Brief: no grant outlives its instance" (sage-2568,
  mos-1271, Vikunja 524).
 - Code read at 47a35bf43: `packages/bus/src/broker.mjs`, `runtime.mjs`, `credentials.mjs`, `host-grant.mjs`,
  `gitea-grant.mjs`, `packages/business/src/instance-settings.mjs`, `tools/host/mosaic-keys-write`.
 - Row 256's delete (honker/row256-r2 at 1e761962a, not landed) changes only the delete's checks. Its write still
  goes through `lifecycleDelete` and the same `revoke()`, so this design covers it unchanged.
 - Row 235 r6 section 6 (host revoke) and #436b r5 section 6 (Gitea revoke and the fire steps).
 ## 1. What happens today (read from the code, to be pinned by a test before the change)
 - **Fire.** `revoke()` (runtime.mjs:151) revokes only the roles entry's `credentials` services, then lists the
  agent's store directory and refuses `credential-revoke-failed` if any file is left. A host grant's key is
  `<store>/<business>/<agent>/host.<id>` (host-grant.mjs:102), and a Gitea grant's token is `.../gitea`
  (gitea-grant.mjs:379). Neither is a roles-entry service, so a fire of an instance with a grant refuses at
  step B with "host.<id> still in the store". It doesn't leak, but it can't finish until a human runs
  `mosaic hosts revoke` or `mosaic grants revoke` by hand. If the business file also names `gitea` as a
  roles-entry credential, `credentials.revoke(instance, 'gitea')` unlinks the granted token file with no
  `gitea_grants` change and no Gitea call.
 - **Delete.** `credentials.remove(instance)` (credentials.mjs:400) revokes every registered reference,
  `host.<id>` and `gitea` included, and unlinks the directory. The `host_grants` and `gitea_grants` records
  stay, the host line stays on each host, and the Gitea token, user and login stay live. This is the leak.
  The next start then finds a grant record whose key file is gone. What the start does with that isn't read
  yet. The T1 test below pins it.
 - **An open grant decision.** A fire delists the instance but keeps its roles entry. An approved host or Gitea
  grant decision for that agent that hasn't run yet still passes `#hostDefinition` and `#giteaRole`, so it can
  still apply after the fire. A delete removes the roles entry, so its executor refuses `unknown-role` or
  `gitea-not-declared`.
 ## 2. Where the grant revokes sit (D1)
 The grant revokes run in the broker, awaited, inside `#chiefLifecycle`, after step A and before step B:
 1. `write('instance.fireCheck')`: the writer's checks against the file, as today. A refusal here revokes
   nothing.
 2. Step A: the runs stop and the open requests are cancelled, as today.
 3. **New, step G:** the grant revokes (section 3). A refusal stops the fire here.
 4. Step B: `write('instance.fire')`. The writer's checks, then `beforeWrite`, which is `revoke()` for the
   credentials, then the write. With the grant files gone after step G, `revoke()`'s store-directory check
   passes.
 5. Step C: as today.
 Why not in `beforeWrite` or `revoke()`: both are synchronous and run inside the organization writer, after
 its checks. A host `remove` is an SSH call and a Gitea revoke is HTTP, so they can't run there. The business
 package also has no reach into the broker's grant tables, and shouldn't get any.
 Why after step A: a fire is the decision to stop the agent. If step G came first, a host that can't be reached
 would keep the fired agent running. After step A, a refused step G leaves the same state as a failed step B
 today: the instance is "firing" (stopped, still listed) until a repeat. Row 194 already handles that state.
 The repeat fire's own path (webby-0094 N1: an instance already delisted still has its tokens revoked, with no
 write) also runs step G. Step G sits in the broker before `write('instance.fire')`, so it runs on both paths.
 The delete: `lifecycleDelete` (broker.mjs:3293) runs step G after its running and starting checks and before
 `instances.set delete`. It becomes async. Its one caller, the launcher's `host.op` (launcher.mjs:845), already
 awaits the reply, and `process.mjs` already returns async ops (`mayLaunch`). A refused step G answers the
 launcher with the refusal. `remove()` returns `{refusal: e.code}`, and the launcher closes the request through
 `lifecycleDone` with that code, as it does today for a writer refusal. The request is then closed, so the
 "repeat" for a delete is the human's new delete request. Grants revoked before the refusal stay revoked, and
 the new request's step G goes on from there. Q2 for Sage: is it fine that a refused delete needs a new request?
 The other choice is leaving the request taken, which today's launcher pass doesn't do for any refusal. Also to
 check at build time: the launcher's `host.op` reply timeout against an SSH remove and the Gitea calls. If it's
 shorter than they can take, the design needs a change before code.
 ## 3. Step G: what it revokes, and how
 The set, for (business, agent), read in one store transaction at the start of step G:
 - each `host_grants` row of the agent;
 - each host named by an open host grant decision for the agent (approved, with no `grant.applied` or
  `grant.failed`, and no `grant.revoked` of (agent, host) after it);
 - the Gitea grant if the agent has a `gitea_grants` row, a `gitea_users` row, or an open Gitea grant decision.
 An empty set means step G does nothing and records nothing. A fire of an instance with no grant is then
 exactly today's fire, with no new events (brief tests, fourth bullet).
 ### 3a. A host grant: the host line first (D2)
 hosts.revoke goes local first: the record drops, the key leaves memory and its file, and only then is
 `remove <user> <tag>` sent. If the host can't be reached, the revoke closes with `hostLineLeft` and the
 record is already gone. The brief says a fire may not drop a grant record while its key could still be on the
 host. So the fire's host revoke reverses one step and keeps the rest the same:
 1. Abort an executor in flight for (agent, host), with no record change (`hostKeys.abortClaim`, as the
   `credential-unknown` branch of `#revokeHost` already does).
 2. `remove <user> <tag>` through the installer, pinned to the host record's keys. `mosaic-keys-write` says
   "a missing tag changes nothing", so this is safe to repeat and safe for a host named only by an open
   decision. If the host record is gone, or the remove doesn't exit 0, step G refuses for this host
   (section 4). Nothing local has changed for this grant: the record, the key file and the memory copy stay.
 3. hosts.revoke's step 1 transaction, unchanged: the `host_grants` row goes and `grant.revoked {revoke, agent,
   host, tag, hostLine: 'pending', granted, reason, by}` is recorded, plus `fire: true` (or `delete: true`)
   so the human can tell it from a manual revoke.
 4. hosts.revoke's step 2, unchanged: `hostKeys.revoke`, so the key leaves memory and its file goes.
 5. `grant.revoke.closed {hostLine: 'removed'}` with no second remove. Step 2 already did it.
 This is one shared method, `#revokeHostGrant(business, agent, host, reason, by, { lineFirst })`.
 hosts.revoke calls it with `lineFirst: false` and its behavior and records don't change (brief item 4). The
 fire and the delete call it with `lineFirst: true`.
 The tradeoff: hosts.revoke goes local first so that "the part that matters can't wait on the network" (r6
 section 6). For a fire, step A has already stopped the agent's runs, and a refused fire keeps the instance
 "firing". So waiting on the host costs nothing in exposure, and the record never lies about the host.
 ### 3b. The Gitea grant (D3)
 The Gitea revoke stays grants.revoke's steps with `fire: true`, which is what `credentials delete` already
 runs (#436b section 6). Step 1 takes the token out of memory and unlinks the file. Step 2 is
 `#giteaRevokeLocal(..., fire: true)`. Step 3 is `#giteaCloseRevoke`: every token deleted and
 `prohibit_login: true`. The fire awaits step 3 instead of leaving it to `setImmediate`.
 The Gitea side can't go remote first the way a host does. The record has to drop and `grant.revoked` has to
 name the tokens before the DELETE calls, or a restart in between would find a live record whose tokens are
 gone (section 5's reconcile). Two changes keep the fire closed:
 - **Preflight.** Before step 1, `#giteaAdminChecked` runs. If the admin token can't be read or checked,
  step G refuses with nothing changed.
 - **Left means not done.** If `grant.revoke.closed` lists anything in `serviceLeft`, step G refuses. The
  token ids stay in the evidence, which is exactly what `mosaic grants sweep gitea` retries, so nothing
  forgets them. The record is gone, but no key of this stack's is still usable: the file and memory copy are
  gone, and only the Gitea side is left. That is the case the brief's host rule guards against, and it is
  kept visible here, not hidden.
 ### 3c. Open decisions
 A host named only by an open decision has no record. Step 3a still runs `remove` (a no-op) and records
 `grant.revoked {granted: false}`, as hosts.revoke does for a revoke with no record (N4). The executor's
 `#revokedSince` then ends that decision `revoked`. The Gitea side is the same through
 `#giteaRevokedSince`.
 ## 4. A refusal (D4)
 Step G stops at the first grant that can't be revoked. Grants already revoked in this step G stay revoked and
 recorded. The rest stay as they were. The fire refuses `grant-revoke-failed`, a new code. The organization
 file is unchanged, since step B never ran, and the credentials are unrevoked, since `revoke()` never ran.
 `BusError` carries only a code, so the names go in one grant-evidence event recorded before the throw:
 `grant.revoke.refused {agent, grant: 'host:<id>' | 'gitea', reason, verb: 'instance.fire' | 'lifecycle.delete',
 by}`. Its `reason` is one of `host-unknown`, `host-remove-failed`, `gitea-admin-unavailable` and
 `gitea-left:<ids>`. A message lands in the human's inbox, as `grant.failed` does today (broker.mjs:1139):
 "Fire of <agent> stopped: its grant on <host> couldn't be revoked (<reason>). Run the fire again once <host>
 is reachable." The refusal record has the verb and args, as every refusal does.
 ## 5. A repeat fire (D5)
 A repeat runs the whole fire again: the checks, step A (nothing left to stop), step G, B and C.
 - A host grant already revoked has no row and no open decision, so it isn't in the set. It's done.
 - A host grant that refused still has its row, so it's retried from 3a step 1.
 - Gitea: if the set is empty but the agent has an entry in `#giteaLeft`, or an open Gitea revoke, step G runs
  `#sweepGitea` for the business first and then checks `#giteaLeft` again. Any entry still there for this
  agent refuses `gitea-left`. The sweep is the human's verb, and running it here records `grant.sweep` and
  `grant.swept` with `by` set to the firing chief. That's an internal caller of an existing method, not a new
  verb. An entry left by an earlier manual `mosaic grants revoke` of the same agent blocks the fire the same
  way. That's intended: no grant outlives its instance, whichever verb started its revoke.
 - A host line left by an earlier manual `hosts.revoke` (`hostLineLeft`, no row) doesn't block the fire. Its
  record is already gone and `mosaic hosts sweep` owns it. The fire's result names it (section 6), so the human
  sees it. **Question Q1 for Dewey and Sage:** should it block instead, as the Gitea leftovers do? Blocking would
  make the two halves match. It would also make a fire wait on a host the human may already have retired.
  I lean toward naming it without blocking.
 ## 6. The result and the event (D6)
 The fire's result and the delete's done record gain `grants: [{grant: 'host:<id>' | 'gitea', revoke,
 granted}]` beside `credentials`. Any leftover from 5's last bullet goes in `grantsLeft`. `fire.done` (step
 C) carries the same `grants` list, so the event the human sees names them. A fire with an empty step G
 carries neither key, so its shape is exactly today's.
 ## 7. What doesn't change (brief item 4)
 - `hosts.revoke`, `grants.revoke` and `grants.sweep`: same callers, the same refusals (`tester-dev-only`
  included), and the same records. hosts.revoke's own order stays local first.
 - No new verb, no Console route, no CLI change, no schema change. Every new record is an `events` row in the
  existing grant-evidence shape.
 - `revoke()` in runtime.mjs and the organization writer don't change.
 ## 8. Tests (each in the owning package, whole files)
 Bus, with the fake installer and the fake Gitea the row 235 and 236 tests use:
 - T1 (pins section 1 before the change): a fire of an instance with a host grant refuses
  `credential-revoke-failed` today. A delete leaves the `host_grants` row. Then: what the next start does with
  a record whose key file is gone.
 - T2: a fire with a host grant on two hosts and a Gitea grant revokes all three. Each gets `grant.revoked` and
  `grant.revoke.closed` as the manual verbs record them, `fire: true`, the line gone on both fake hosts, every
  token deleted and login prohibited at the fake Gitea. Afterwards that agent's `host.run` and Gitea use both
  refuse.
 - T3: the same through the delete (`lifecycleTake` then `lifecycleDelete`).
 - T4: host B unreachable. The fire refuses `grant-revoke-failed`, and `grant.revoke.refused` names the agent,
  `host:B` and `host-remove-failed`. The file is unchanged. Host A is revoked and recorded, host B's row and key
  file are still there, the Gitea grant is untouched (it comes after B in the order), and the credentials are
  unrevoked. A repeat with B reachable completes and revokes only B and Gitea.
 - T5: Gitea's admin token can't be checked. The fire refuses with the Gitea grant intact.
 - T5b: a token DELETE fails. The fire refuses `gitea-left`. A repeat with Gitea back sweeps and completes.
 - T6: an approved host grant decision that hasn't run yet, for the fired agent: the fire records
  `grant.revoked {granted: false}`, and the executor then ends that decision `revoked`.
 - T7: an instance with no grant fires exactly as today. Same result keys, and the same events, counted.
 - T8: hosts.revoke and grants.revoke keep their records, their order and their tester refusals.
 Business: none expected, since the writer doesn't change. Cli: the launcher's delete pass with a refused step
 G closes the request with `grant-revoke-failed`, if the launcher's tests stub `lifecycleDelete`.
 ## 9. Mutants
 - M1: step G skips the host revokes. Killed by T2.
 - M2: step G skips the Gitea revoke. Killed by T2.
 - M3: a refused grant revoke is ignored and the fire goes on to step B. Killed by T4.
 - M4: `lifecycleDelete` skips step G. Killed by T3.
 - M5: the fire's host revoke goes local first (`lineFirst: false`). Killed by T4, because B's row is gone.
 - M6: a Gitea revoke with something in `serviceLeft` counts as done. Killed by T5b.
 - M7: open decisions left out of the set. Killed by T6.
 ## 10. Order and rebase
 Built from 47a35bf43 for this read. The code goes on whatever refactor is when the build starts. Rows 274, 256
 and 194's follow-ups touch the broker and instance-settings, so expect a rebase and a whole rerun. The row lands
 before the .53 Case 1 install hires an instance with a grant. It doesn't hold the dragon-lin re-pin.
 ## Runs
 (after the gate)
 ## Mutants
 (after the gate)