Files
jason.woltje b1fece6f0a row 235: machine grant row b, raise binding, executor, reconcile, revoke and sweep, creative roles refused at raise, recheck and host.run, N10 (#1741)
Lands Gizmoduck's round 1 candidate, ed4193316 on gizmoduck/row235-e (one commit on 62ddc96cc), approved
by Duckworth (duckworth-0089, REVIEW-r1 d9ecec514 on duckworth/reviews, #1741 comment 30022). 13 paths,
manifest dad377ca8fe4. No 235 path changed between 62ddc96cc and this commit's parent. Patch-id
c355d2fc457d, the same as 62ddc96cc..ed4193316. broker.mjs keeps row 213's import and decision.raise
creative refusal as landed.

Creative roles (mos-1163, mos-1164, sage-1908): #hostDefinition refuses host-creative-role at the raise
and at the step 1 recheck, and #hostAccess refuses it at host.run before reading the grant record, which
stays for hosts.revoke. Duckworth checked from the code that a creative role can't raise, approve or
spend a host grant, and ruled the upsert mutant equivalent.

Evidence: Gizmoduck's runs on my slot. B1-B37 on 5df4b52c5, and run 2 on 534f2254a (10:12:52-10:14:51Z):
grant-host 9/9, grant-executor 37/37 and grant-enroll 24/24, each run whole, with B16, B23, B27, B38 and
B39 killed. B1-B39: 38 killed, and B17 is equivalent (sage-1904).
Gate on my slot, 10:30:55-10:42:43Z on 29ce1e360 plus the 13 paths: bus 596/0 (its first whole run on
this head, duckworth-0089 N1), business 200/0, cli 666/0, and every scripts/test-*.sh green (auth 15,
conductor 17, config 31, discord 78, extension-package 18, foundation 44, land-work 86, queue 156 and 28,
release 14, task 98, all 0 failed).
2026-10-21 05:42:52 -05:00

39 KiB

Row 235 build: machine grant row b (#436, Gitea #1741)

Brief: docs/plans/2026-10-20_host-credentials.md, "Machine grant row b: raise binding, executor, reconcile and revoke". Design r6 f9039682e. Built on row 234's landing c771a36f4 (sage-1775), branch gizmoduck/row235-b. Line numbers below were read on b0320d395, whose src files are byte-identical to the landing's.

Squash onto 4ced5e63d (sage-1812, sage-1831)

Steps 1-6 were six local commits on c771a36f4 (gizmoduck/row235-b, tip f9fff4300). Sage named 4ced5e63d as the base, so this branch, gizmoduck/row235-c, holds one commit on it: the diff c771a36f4..f9fff4300 applied with git apply --index. N10 is in that commit. N12 needs no change, because row b doesn't touch schema.sql.

Rows 240, 228 and 238 landed between c771a36f4 and 4ced5e63d and changed 18 paths. One of them is also mine:

  • packages/cli/README.md. Row 228 (86249ac55) changed the session credential paragraphs at about lines 432 and 706. My hunk adds three lines on hosts revoke and hosts sweep at about line 1052. The two don't overlap, so the blob here is row 228's blob plus my three lines, with no landed line reverted. cli tests/host.test.mjs:855 reads this README, and run 1's cli suite covers it.

The other 12 paths are byte-identical to f9fff4300. The line numbers in the step notes below were read on the old base.

Run 1 (sage-1831, sage-1833, sage-1838)

Run 1 on 4d4bb4af6, 07:45-07:54Z, failed: 1351 tests, 1285 pass, 66 fail. The 66 are 33 tests counted twice, once in the files run and once in the bus suite. The cli suite, the cli hosts test, seat sandbox.test and bus project-runtime.test passed. Two defects, both mine:

  • grant-executor.test.mjs, 32 tests. setup() never made dir/data, and new Store lstats its root (store.mjs:137), so every test stopped at setup with ENOENT. setup() now makes it, mode 0700.
  • credential-claim.test.mjs:76 (X4), 1 test. A register after a revoke left the key in #revoked, so status() said revoked for a new grant's reference that has no token yet. register now clears #revoked, as unregister does. #revoked feeds only status(), and issue clears it as well.

Both fixes are amended into the one commit. The rerun covers the five bus files, the bus suite, and cli credentials.test.mjs and verbs.test.mjs, which read credential state strings.

Run 1 rerun (sage-1842, sage-1843)

The rerun on ac7c45975, 08:06-08:07Z, failed: 701 tests, 695 pass, 6 fail, which are 3 grant-executor tests counted in both the files run and the bus suite. All three were test defects that run 1's setup failure had hidden. No src file changed for them:

  • G10. The two fixture decisions were built after before, and grantDecision (host-fixtures.mjs) INSERTs one events row each, so the count read 13 for 11. The test now builds them first, asserts they added exactly 2 rows, and then pins the executor's 0 against that count.
  • G27. setup's t.after removes the scratch dir before a hook the test adds later runs, so the test's chmod hook hit ENOENT after the body had passed. The body now restores the mode in a try/finally.
  • G22. The slow run's promise had no handler until refusesWith, four lines on. The revoke rejected it first, and node failed the test on the unhandled rejection. The handler now goes on at creation.

The fix is one more amend of the single commit. The rerun is grant-executor.test.mjs whole plus the bus suite.

Rerun 1c on 5df4b52c5, 08:15Z, was green: 623 of 623, grant-executor 34 of 34 and the bus suite 589 of 589.

Mutants B1-B37 (sage-1894, sage-1899, sage-1904)

Three chunks ran on 5df4b52c5, 09:22-09:37Z. Restores matched HEAD after each. 33 were killed and four survived. Two of the misses were mine, in the file lists:

  • B2 and B23 listed only grant-executor, and that file never makes the raise either mutant changes. grant-enroll.test.mjs:227 (A2, A4) resolves a plain credential.mint yes and a host grant no, and asserts no grant events for either. mutants.py now lists grant-enroll and grant-executor for both. B2 then ran alone and A2 killed it. B23 runs again with this round's tests.

The other two survivors needed tests, and Sage ruled on all four (sage-1904):

  • B16 drops the abort check after verify. The check after step('commit') still ends the grant revoked, so G23 X5's asserts held. The run then logs a commit step for a grant that never committed. G23 X5 now asserts that its only grant.step is install.
  • B17 adds await null after the commit check. Sage accepted it as equivalent. await null yields only to microtasks. Every revoke starts in a macrotask of its own (a socket request, or a test's call), so none can land in that gap. There is no test for it.
  • B27 makes issue's commit callback async. issue() doesn't await it, so the executor returns applied before the INSERT runs. The two differ only when the commit transaction throws, and a real store can throw there with a full disk, an I/O error or a busy timeout. Under the mutant that throw becomes an unhandled rejection after the key file is written, and the catch's undo never runs. The new G8 test makes the store's run throw SQLITE_FULL once on the host_grants INSERT. It asserts host-failed, no record, key file, reference or line, and the steps install, commit, failed. The store object comes from the test's setup, so src gains no seam.

These two test changes went in as a new commit on gizmoduck/row235-d (8b13d0ca0). That branch then moved onto row 213's landing as gizmoduck/row235-e, below. B16, B23 and B27 run there, after the touched test files run whole.

Rebase onto 62ddc96cc (sage-1922)

Row 213 landed as 361ba2387, and refactor's tip is 62ddc96cc, with only docs past it. gizmoduck/row235-e starts at 62ddc96cc and cherry-picks 8b13d0ca0's one commit. The patch-id is unchanged (c062b73fa9f1). row235-c and row235-d stay as they were.

Between 4ced5e63d and 62ddc96cc, 361ba2387 is the only commit that touches a file of mine. The one drifted path is packages/bus/src/broker.mjs:

  • 213 adds CREATIVE_DEFINITIONS to the vocabulary import at :6, and refuses decision.raise from a creative role with decision-not-allowed at :2176. Both hunks stay as landed.
  • My 31 removed broker lines read the same before and after the cherry-pick, so no landed line is reverted.

packages/cli/README.md isn't in 361ba2387. Every other path of mine is byte-identical to 8b13d0ca0. Line numbers in the section below were read on the row235-e working tree; earlier sections keep theirs.

Creative roles get no host grant (mos-1163, sage-1905, sage-1908, sage-1916)

Brief blob b16e236188 (4364dddb0, rev 2784) requires two refusals: the writer, at the raise and at the step 1 recheck, and host.run at spend time, with the record kept. Webby's N2 on row 213 r4 asks whether a creative role can spend a decision another role raised.

The spend answer. Mos confirmed it read-only (mos-1164, sage-1916) on 8b13d0ca0 and 01e1fe512; the row235-e line follows each:

  • Only the chief raises a host grant. #raiseGrant refuses grant-chief-only for any other definition: :1232 on 8b13d0ca0, :870 on 01e1fe512, :1238 here. The hosts verbs check the same at :1307 on 8b13d0ca0, :896 on 01e1fe512, :1313 here. chief-of-staff isn't in CREATIVE_DEFINITIONS.
  • No agent spends one. #consumeGrant (:697) is called for a host grant only from #runHostGrant (:902) and from the restart reconcile (:1106), which ends it interrupted. #runHostGrant starts from the resolve hook (:2881) only when #approvedGrant returns a grant, and that needs a close by the business's human, via cli, with the raise's approval choice (:691). An agent's resolve records via broker, so it starts nothing (grant-enroll A3). The public runHostGrant (:891) is a trusted API that no verb reaches.

So a creative role can't raise, approve or spend a host grant. The gap was a different one: the chief could raise a grant whose agent is a creative role, and the human could approve it. The only check on that agent was #hostDefinition, which refuses host-not-declared (:864) unless the definition's credentials names host. Row 213's checkCreative (business/src/role.mjs:143) limits a creative role's tools, authority and messaging, not its credentials, so a creative role file naming host passed both the raise and the step 1 recheck.

As built:

  • Writer. #hostDefinition refuses host-creative-role (:862) before the host-not-declared check. The raise binding (:1266) and the step 1 recheck (:879) both call it, so a grant raised before its agent's instance moved to a creative definition ends at step 1, spent, with no remote step.
  • Spend. #hostAccess refuses host-creative-role (:714) when the agent's current definition is creative, before it reads the record. Both callers go through it: host.run (:2543) and the host.sudo raise (:2778). The record stays, because revoke still owns the cleanup (sage-1908). This goes past the brief's writer line in the same direction; Sage tells Mos.

Tests:

  • grant-host.test.mjs:115. A muse role on a research definition that declares host. The raise refuses host-creative-role and adds no decision.
  • grant-executor.test.mjs:947. The restart moves ops to a creative definition that declares host, then a fixture grant runs. It ends host-creative-role at step 1, the decision is spent, and there is no remote request and no record. The fixture comes after the restart, because the restart's reconcile ends any resolved, unspent host grant as interrupted.
  • grant-executor.test.mjs:958. An applied grant, then a restart with ops creative, then host.run refuses host-creative-role. No remote request is added, sudo.log has no systemctl, and the record is still there.

The broker clones businesses when it's built, so a definition change needs a new Broker. The test's restart() gains a business parameter, default unchanged, for that.

Mutants: B38 removes the #hostDefinition line, run against grant-host and grant-executor. B39 removes the #hostAccess line, run against grant-executor.

Run 2 (sage-1928)

Run 2 on 534f2254a, 10:12:52-10:14:51Z, was green. grant-host.test.mjs 9 of 9, grant-executor.test.mjs 37 of 37 and grant-enroll.test.mjs 24 of 24, each run whole. grant-enroll is unchanged, but it drives the broker, and broker.mjs changed. Then the five mutants, all killed, with the restore matching HEAD:

Id Killed by
B16 G23 (X5), the revoke as verify returns
B23 A2, A4
B27 G8, the store fault in the commit transaction
B38 both mos-1163 tests
B39 the sage-1908 test

Mutant results, B1-B39

Run Head Killed Survived
B1-B13, 09:22Z 5df4b52c5 12 B2 (file list)
B2 alone, B14-B37, 09:28-09:37Z 5df4b52c5 21 B16, B17, B23, B27
B16 B23 B27 B38 B39, 10:13Z 534f2254a 5 none

38 of 39 are killed. B17 is equivalent (sage-1904), for the macrotask reason in the B1-B37 section. B1-B37 ran on 5df4b52c5, before the move onto row 213's landing, and weren't rerun on 534f2254a. --check matches all 39 anchors once on 534f2254a.

Carry list

  • N10 (duckworth-0084, row 234 r1): cli/src/hosts.mjs:30-31 reads shape[verb] off a plain object, so mosaic hosts constructor passes usage and the bus refuses it with exit 2 instead of 4. Row b adds hosts revoke and hosts sweep to the same table (design r6 :562, :680), so the fix goes in with them: Object.hasOwn(shape, verb) and a usage test for an Object.prototype name (sage-1726).
  • Untimed tables (sage-1745, row 234 r3): stack-release T46 lists hosts and host_grants as untimed because every write to them shares one store transaction with a grant.* events row. Row a writes only hosts. Every host_grants INSERT, UPDATE and DELETE row b adds must do the same, and BUILD.md lists each with its event line. A write without one is a stop to raise with Sage.
  • N12 (duckworth-0086, row 234 r3, doc-only): row 234's round 3 list of other schema hits leaves out packages/cli/tests/stack-release-fixture.mjs:214 and scripts/test-land-work.sh:254. Both look up one table by name and need no change. If row b changes the schema, its grep list names them.

What row a already gives row b (b0320d395)

  • #approvedGrant, #consumeGrant(business, decision, method) and GRANT_EVIDENCE (broker.mjs:667-690). Row b calls #consumeGrant with 'host'.
  • #openGrantDecisions(business, method) and reconcileGrants() (broker.mjs:823-860), which today handle host-enroll only. Row b adds the host method, the grant.step undo table (r6 X3) and the revoke-marker reconcile (r6 section 6, N4).
  • The resolve hook at broker.mjs:2444-2445 starts #runEnrollment for host-enroll. Row b adds the same start for host, under the same human, via cli and choice checks.
  • host.run already reads the store grant record and checks the Approve record (broker.mjs:691-703).
  • The raise check grant-chief-only (broker.mjs:870, :896). host-grant.mjs:55 refuses every method but host-enroll, and row b opens it to host.
  • Credentials has #inflight with the host-busy claim inside runHost (credentials.mjs:118, :355). It has no claim, abortClaim, register or unregister. Row b adds those (r6 X4, X5).

Plan, in build order

  1. Raise binding (r6 R1-R4): the grant argument for method: 'host', host-not-declared, host-not-ready, the 210 D6 argv rules, rotateBy, the rendered block and the 4096 cap, and the raise body hashes. Tests G1-G5, G19.
  2. Credentials.claim, release, abortClaim, register, unregister, with the shared reference check.
  3. The executor in bus/src/host-grants.mjs, steps 1-6 (r6 X2), with failure undo (X3) and the abort rule (X5, C1). Tests G7-G10, G15, G23, G24.
  4. Restart reconcile for grant.step, unspent host decisions, and the start sweep of host.<id> key files with no record (r6 X3, N1). Tests G11, G27, G28.
  5. hosts.revoke and hosts.sweep verbs, plus the CLI revoke and sweep with N10. Tests G12, G13, G26, G27.
  6. G22 end to end on one bus. Then the mutant list from r6 section 10 for row b, numbered B1 onward in this file, with the upsert mutant and the abort-after-commit mutant included.

There is no schema step. If one turns out to be needed, I stop and ask Sage to re-brief first.

Step 1 notes (raise binding, read on b0320d395)

Where the code goes:

  • #raiseGrant (broker.mjs:866-885) runs the checks and keyscan for host-enroll today, and every raise with grant reaches it (:1738-1739). Row b branches on a.grant?.method. host takes no keyscan, so its checks can all run inside the one raise transaction, as R4 asks.
  • enrollCheck (host-grant.mjs:50-69) refuses every method but host-enroll (:55). Row b adds a sibling hostGrantCheck for the R4 shape {method, agent, host, commands, rotateBy} and keeps enrollCheck as it is, so the 234 tests don't move.
  • The raise body is at broker.mjs:2270, :2345-2353 and :2391. enroll becomes a generic grant binding. The question block for host follows the host.sudo block (:2337-2341): agent, host id, address, user, sudo, each prefix JSON-quoted, rotate date, and new key or changed commands. The 4096 cap comes from string(question) (:2354).
  • The argv rules come from checkCommands (business/src/hosts.mjs:67-87): no wildcard, no empty prefix, no glob characters, a bare name or an absolute path, and refusedCommand for shells and runners. It throws refuse(...) with a message, so the bus maps any throw to invalid-request, as enrollCheck maps checkAddress (host-grant.mjs:57-61).
  • host-not-declared: the agent's definition must list {"service": "host"}. business.mjs:126-129 reads definition.credentials and filters out HOST_SERVICE (vocabulary.mjs:113). The bus needs the definition's service list. Read how the running business exposes the definition before choosing.
  • Both lookups are answered. loadBusiness returns definitions, each role file's validated document (business/src/business.mjs:114-124, :159, returned at :267). busBusiness clones the whole business (bus/src/business.mjs:7), so the broker has b.definitions. #definitions already reads it (broker.mjs:1027-1029), and the comment there says the broker doesn't read roles/ (sage-0590).
    • host-not-declared checks b.definitions[b.roles[agent].definition].credentials for {service: 'host'}. role.mjs:118 normalizes a host entry to exactly that.
    • Deviation from R4: definitionSha256 is sha256 over canonicalJson of that validated definition, not over the file's bytes. The broker doesn't read roles/. A whitespace-only edit to the file no longer voids an approval, but any change to a field the loader keeps still does. The bus sees a definition change only after a business reload or restart, the same as every other role field. This goes in the round 1 note.

Assumption, accepted for round 1 by Sage (sage-1782), for Duckworth to check in review: design R4 says rotateBy is "a date inside 210's rotation window", but 210 defines no window in code. It checks only that the value is a real calendar day (bus credentials.mjs:16-20, issue at :245). The business load calls a past rotateBy a problem (business/src/credentials.mjs:114). So the raise takes a real day strictly after today, UTC, and refuses invalid-request otherwise. There is no upper bound, because no requirement states one and 210 checks none.

Step 1 as built (not run yet; no slot)

  • hostGrantCheck(grant, today) in host-grant.mjs takes exactly method, agent, host, commands and rotateBy. agent must match ID_PATTERN and host HOST_ID. commands goes through checkCommands, and any throw becomes invalid-request. rotateBy must pass 210's day (now exported from bus credentials.mjs) and be later than the bus clock's UTC day.
  • #raiseGrant branches on a.grant?.method === 'host' after the chief check and before enrollCheck. So a host-enroll shape with method: 'host' still refuses invalid-request, and the 234 G1 list holds.
  • #hostGrantBinding runs inside the raise transaction. It refuses unknown-role for an agent that isn't a role instance (the code bindLaunch uses, broker.mjs:522). It refuses host-not-declared, and host-unknown for a host with no record, as #hostAccess does. A record in pending-install refuses host-not-ready. R4 names only host-not-ready. I picked host-unknown for a missing record so "not enrolled" and "enrolled but not checked" stay apart.
  • commandsSha256 is sha256 over JSON.stringify(commands). For arrays of strings this equals canonicalJson, and step 3's executor recomputes it the same way.
  • #dispatch takes binding = {body, block} in place of enroll. The enrollment block moved into enrollBinding at module level with its text unchanged, so the 234 G4 test reads the same question.
  • No new import in broker.mjs, so RUNNER_FILES doesn't change. host-grant.mjs now imports business/src/util.mjs for canonicalJson. Nothing in the sandbox loads host-grant.mjs. Before in-review I still run seat sandbox.test.mjs and bus project-runtime.test.mjs whole.
  • The new-key line reads host_grants (a SELECT). Step 1 writes no host_grants row.
  • Tests: bus/tests/grant-host.test.mjs, G1 (host), G2, G3, rotateBy, G19, R4/G5 body hashes, G4 block. The G5 tester-thread case: a thread run is always THREAD_DEFINITION, never chief-of-staff, so grant-chief-only refuses it first. The test covers the tester's own cap (tester-refused).

Step 2 as built (not run yet)

  • The constructor's per-reference checks moved into #checkRef, in the same order, and register calls it (r6 X4). register reads no file. A key file that a failed revoke unlink left at the path stays unused until the start sweep (step 4) removes it. register refuses credential-reference while the entry holds a live token. An entry with no token, revoked or never issued, is replaced, so a new grant after a revoke registers again.
  • unregister drops an entry with no token and clears its revoked mark. A live token refuses.
  • claim refuses host-busy when the slot is held, by a claim or a host.run, and returns the signal. It needs no reference, because the executor claims at step 1 and registers at step 5. release(instance, service, signal) frees the slot only for its own signal. I added the signal argument so a late release can't free a later claim.
  • abortClaim aborts whatever holds the slot and returns {pending}. An aborted claim keeps the slot until the executor releases it in its finally (X2).
  • Tests: bus/tests/credential-claim.test.mjs. The D5 use( allowlist is unchanged.

Step 3 as built (not run yet)

Where the code goes. The remote steps are in host-grant.mjs (grantKey, install, verify, remove) and the orchestration is #runHostGrant in broker.mjs, next to #runEnrollment. BUILD.md's plan said a new bus/src/host-grants.mjs. I kept row 234's split instead: the broker holds every store write and host-grant.mjs holds the network and key files. A new broker import would also change what the sandbox must mount (RUNNER_FILES), and this way broker.mjs gains none.

The broker gets hostKeys, made by grantKeys({credentials, storeRoot}) in host-grant.mjs. It wraps only claim, release, abortClaim, register, unregister, issue and revoke, keyed by (business, agent, hostId). The broker still never gets use(). runtime.mjs builds it and passes it in. hostReference gives the store reference, and the runtime's start load uses the same function.

Every host_grants write in step 3, with its event (sage-1745):

Write Where Event in the same transaction
INSERT, a new key the commit callback that issue calls grant.applied
UPDATE commands and decision, a commands change step 5's transaction grant.applied

No other step 3 code writes host_grants.

Deviations from r6, for the round 1 note:

  • grant.applied commits with the record, inside commit, not after issue at step 6. The untimed-table rule needs an event in the record's transaction. grant.step commit can't be that event, because X2 writes it before the key file. So X3's "5 after commit" row has nothing to undo in the executor: once the record commits, grant.applied has too. What's left in issue after commit is the memory fill, which writes no file and no store row. The executor sets committed after the transaction returns and from then ignores an abort (C1). The restart reconcile (step 4) keeps the commit row, because a crash can fall between the key file write and commit.
  • Event bodies name the decision as grantDecision, as row 234's enrollment events do. The consume record is the only grant body with decision, so #spent counts it and nothing else.
  • For a commands change, grant.applied has fingerprint: null. The executor holds no key on that path. The record's rotate_by stays. X2 names only commands and the decision id.
  • The keygen dir goes as soon as grantKey has read the key back into memory, not in step 5's finally.
  • Step 4 runs true as the remote command as is, not through the sudo argv, since a user with no sudo must pass it too.
  • hostGrant() takes installerUser, mosaic-installer by default. Only the tests change it, because the scratch sshd logs in only as the test's own user.
  • A commands-change UPDATE that changes no row ends host-changed. Only a revoke deletes a live record, and a revoke aborts the claim first, so this is a guard.

Step 1 codes, after the spend and in its transaction: host-not-configured, unknown-role, host-not-declared, decision-mismatch (definition, registry or commands hash), host-unknown, host-not-ready, revoked (a grant.revoked of the agent and host after the raise). The definition check moved into #hostDefinition, which the raise binding uses too.

Tests: bus/tests/grant-executor.test.mjs, a scratch sshd on 127.0.0.1 with stub list, add and remove that edit a second authorized-keys file. G7 (new key, and a commands change), G8 (install, verify, a failed remove, step 5 before commit, a failed step 1 check), G9, G10, G15, G23 (a revoke during install through abortClaim, a grant.revoked before start, and a held slot), G24's tag part. G23's held slot uses claim in place of a running host.run. Both hold the same #inflight entry. The revoke verb comes in step 5, so G23 here writes grant.revoked through the fixture writer and aborts through abortClaim. Step 5 adds the verb's own path.

Step 4 as built (not run yet)

The restart reconcile has three parts, in start order:

  1. sweepHostKeys (host-grant.mjs), in runtime.mjs right before Credentials loads. In each loaded business's agent store it unlinks every host.<id> file that has no host_grants record, and logs one hosts: {"event":"host-key-swept",...} line per file to stderr. A store dir that isn't private refuses credential-file, and so does the start. A name that isn't host. plus a valid host id stays, so ssh-keygen's .pub stays too. A business the bus didn't load is left alone.
  2. reconcileGrants, where row 234 already calls it, after Credentials loads and before the socket opens. For a host grant it ends a spent decision that has no grant.applied and no grant.failed. It also spends and ends an unspent one the human resolved yes. Each ends grant.failed interrupted, with the tag and a notice. When the last step is install, or commit with previous: null, a line may be left. For those, it claims the (agent, host.<id>) slot and queues the decision for part 3.
  3. reconcileHosts, which runtime.mjs calls once serve returns, with no await, so a start doesn't wait on the network. Each queued decision runs remove <user> <tag> through the installer, as the step's user. Then it ends grant.failed interrupted, with hostLineLeft if the remove failed or there's no host record, and releases the claim. The claim makes a new grant for the same agent and host refuse host-busy while the remove runs. Otherwise its new line, with the same tag, could be removed.

Deviations from r6's restart table, for the round 1 note:

  • No record write on restart. Step 3 commits the record with grant.applied in one transaction. So a decision with no grant.applied has no record. For a new key, the record undo has nothing to delete. For a commands change, the "back to previous" undo has nothing to put back. What's left of the commit, new key row is the key file, written before commit, and the start sweep unlinks it. The commit, commands change row ends interrupted with nothing undone. G28's "a revoke before grant.applied" can't happen, because they commit together. Step 5 tests G28 as a change, then a revoke, then a restart.
  • The local undo of row 234's enrollments stays after Credentials loads, where row 234 put it. That undo touches only the installer key and the host record, and Credentials reads neither.
  • The host grant part writes no host_grants row, so the untimed-table rule (sage-1745) adds nothing here.

Fixture change: hostGrantRecord (tests/host-fixtures.mjs) writes grant.applied after a grant.host spend, as the executor always does. Without it, a start over a fixture record ends that record's decision interrupted. Row 234's M3 start test (host-run.test.mjs:602) does that start. No test counts the fixture's grant events (checked by grep across the bus tests).

Tests in grant-executor.test.mjs, through a restart helper that follows runtime.mjs's order on the same store:

  • G11 after install: the line goes, the slot is held between the two parts, and the grant ends interrupted. A second start adds nothing, and runHostGrant refuses decision-consumed.
  • G11 after a new key's commit with the key file written: no key file, record, reference or line.
  • G11 after a change's commit: the record, key and line are unchanged, and the host gets no request.
  • G11 (B2) before step 1, spent and unspent: each ends interrupted with a notice, and the unspent one is spent exactly once.
  • A failed remote undo, and a missing host record: each ends with hostLineLeft and the sweep hint.
  • A finished grant: the reconcile adds nothing.
  • N1 (G27's sweep part): only files with no record in loaded businesses go, and a dir that isn't private refuses.

The revoke-marker part of the reconcile (section 6, N4) comes with the revoke verb in step 5.

Step 5 as built (not run yet)

Two new bus verbs, hosts.revoke and hosts.sweep, join HOSTS_VERBS. The CLI gets mosaic hosts revoke AGENT HOST --reason R and mosaic hosts sweep HOST.

hosts.revoke {agent, host, reason} follows r6 section 6:

  1. One transaction deletes the host_grants record and writes grant.revoked with a new revoke id (a UUID), the computed tag, hostLine: "pending", granted, reason and by. With no record it still writes the marker, with granted: false, so a stale line can be named (N4). A host with no record refuses host-unknown and writes nothing.
  2. With no await between, Credentials.revoke drops the key from memory, aborts a host.run in flight and unlinks the file. With no reference (credential-unknown), an executor holds only a claim, so abortClaim stops it. Any other throw, such as an unlink that fails, comes back as keyFile: "left". The record is already gone, so the next start's sweep unlinks that file.
  3. remove <user> <tag> through the installer, as the host record's user (the executor's user too). Then grant.revoke.closed with the same revoke id and hostLine: "removed", or hostLineLeft.

Callers (G26): the human, and the assistant of the session's business, checked with #threaded. The tester can call it when bus.tester is exactly true, read again at the call. Any other agent gets revoke-refused, a code new in this row. A tester session can't hold a business whose bus.tester isn't true, since bindTester refuses tester-off, and the broker's businesses don't change while it runs. So tester-dev-only can't be reached from a test. The check stays for N8, in case a later row reloads businesses.

hosts.sweep {host} is the human's or the tester's; an agent gets human-required. It runs list <user>, then removes each listed tag that this business recorded for the host in a grant.step or grant.revoked and that no live grant holds. Another business's line and an unrecorded mosaic- line stay (G24's sweep part). It writes grant.host.swept {host, removed, left, by}, where left is each earlier left tag still listed and not removed. #hostLinesLeft now starts from the last grant.host.swept and adds only later hostLineLeft bodies (F4).

The restart reconcile adds the marker part: reconcileGrants collects each grant.revoked with hostLine: "pending", a text revoke id and no grant.revoke.closed of that id. reconcileHosts runs step 3 for each. A grant.revoked with no revoke id, as G23's fixtures write, isn't reopened. hosts.status adds revoking, the tags of the open markers on the host.

Other changes:

  • N10: the CLI checks the verb with Object.hasOwn, so constructor, toString or __proto__ is a usage error.
  • --reason is required with revoke and refused with any other verb.
  • The reason goes through the broker's string(x, 500), as credentials.revoke's does, so a reason of only spaces is accepted there too.
  • Correction to step 4: the first G11 test expected list then remove. reconcileHosts sends only remove, so the test now expects that.

Residuals for the round 1 note:

  • A new grant whose line goes in while a revoke's step 3 or a sweep runs can lose that line, if it has the revoked tag. Revoke holds no claim during step 3. If the remove lands after the new grant's verify, the grant is recorded but its line is gone, so its logins fail. I didn't add a claim, because a revoke must not wait on a slot.
  • The enrollment's installer tag is never in the sweep's set, so a hostLineLeft naming it stays until the human removes it by hand.

Tests in grant-executor.test.mjs:

  • G12: the human's revoke of a live grant, and the bodies of both markers. A restart adds nothing.
  • G12 (N4): a revoke with no record removes the computed tag's line. Bad args refuse invalid-request and an unknown host host-unknown.
  • G13: a revoke during install aborts the executor, which ends revoked. A raise before a revoke and started after it ends revoked at step 1.
  • G26: who may revoke and who may sweep.
  • G27: a restart after step 1 (the sweep takes the key file, then step 3 runs), after step 2 with the host down (hostLineLeft, then a sweep clears it), and a file a revoke couldn't unlink.
  • G28: a commands change, then a revoke, then a restart.
  • G24 (B4): the sweep's tag set, a failing list, and the left list it records.

The CLI tests in packages/cli/tests/hosts.test.mjs cover revoke and sweep, the N10 usage cases and the revoking field in a status line.

Step 6 as built (not run yet)

New tests in grant-executor.test.mjs:

  • G22 (B1), one bus with no restart. The chief enrolls web1, the human approves, and hosts.check marks it ready. A grant of echo follows, then a host.run of echo hi that prints hi. A sleep raise refuses command-not-granted, and after a commands change to echo and sleep the next host.run of sleep 30 starts. A revoke kills that run with credential-unavailable. The next call refuses host-not-granted.
  • G12: the local steps finish before the host step. With the host's remove held, the record and key file are already gone and hosts.status shows the tag under revoking.
  • G13: a revoke whose host remove fails returns hostLineLeft, drops the record and key, and closes the marker, so a restart adds nothing.
  • G23 (X5): a revoke right after the executor's spend reaches its claim and the grant ends revoked.
  • G23 (X5): a revoke that lands as verify returns ends revoked with no record, key file or line. Only the abort check after verify catches it.

The fixture gains a sudo stub (logs to sudo.log, then runs the command as the test's user), a hold-remove flag, opts.enroll (no host record and no installer key) and opts.wrap (changes the host API the broker gets). hostGrant gets a fake scan that returns the scratch sshd's host key.

G22 deviations, for the round 1 note:

  • checkAddress (business/src/hosts.mjs:62) refuses this machine's address, so the test enrolls 192.0.2.10 with the fake keyscan. It then points the record at the scratch sshd with an SQL update and adds the enrolled installer key to the sshd's authorized keys. Enrollment, check, grant, run and revoke all go through the real code after that.
  • r6 B1 says the next host.run after a revoke refuses credential-unavailable. Here it refuses host-not-granted, because step 1 deletes the record and requestHost checks the record (#hostAccess) before it reaches the key. The call in flight does refuse credential-unavailable.

Mutants

/mnt/storage/scratch/gizmo-235-out/mutants.py holds each mutant as anchored edits. --check applies them in memory and runs no test. All 39 anchors match once. Each mutant runs its named test files.

From r6 section 10, as they apply to row b:

Id Mutant Test file
B1 drop the chief check on a host grant raise grant-host (G1)
B2 start the executor on a credential.mint with no grant grant-enroll (A2), grant-executor
B3 skip the ready check at the raise grant-host (G19)
B4 skip the ready check at step 1 grant-executor
B5 skip the definition hash compare grant-executor (G9)
B6 drop the definition check (host-not-declared) grant-host (G2), grant-executor (G9)
B7 skip the commands hash compare grant-executor (G9)
B8 skip the host record hash compare grant-executor (G9)
B9 write the grant record before install grant-executor (G8)
B10 skip remove on a verify failure grant-executor (G8)
B11 skip the start sweep of key files with no record grant-executor (N1, G27)
B12 skip the reconcile's remote remove grant-executor (G11)
B13 skip the grant.step before install grant-executor (G11)
B14 revoke leaves the grant record grant-executor (G12)
B15 skip register before issue grant-executor (G7)
B16 drop the abort check after verify, the last one before issue grant-executor (G23 X5)
B17 add an await between the abort check and issue grant-executor
B18 revoke ignores a claim with no reference grant-executor (G13, G23 X5)
B19 build the tag without the business grant-executor (G24)
B20 sweep removes any mosaic- line grant-executor (G24 B4)
B21 run revoke's host step before the local steps grant-executor (G12)
B22 start the executor on an arbiter resolve grant-executor (G10)
B23 start it on no grant-enroll (A2, A4), grant-executor
B24 write the private key into the event body grant-executor (G15)
B25 put an await between the spend and the claim grant-executor (G23 X5)
B26 skip the start check for a resolved, unspent grant decision grant-executor (G11 B2)
B27 make commit async grant-executor (G8)
B28 route hosts.revoke to Credentials.revoke alone, as credential-verbs.mjs would grant-executor (G12, G27)
B29 skip the revoke marker reconcile grant-executor (G27)

Row 235's own:

Id Mutant Test file
B30 skip the grant.revoked check at step 1 grant-executor (G13, G23)
B31 the sweep set keeps a live grant's tag grant-executor (G24 B4)
B32 reopen a grant.revoked with no revoke id grant-executor (G27)
B33 step 3 reports removed when the host remove fails grant-executor (G13, G27)
B34 any agent revokes, not just the assistant grant-executor (G26)
B35 hostLineLeft ignores the last sweep grant-executor (G24 B4, G27)
B36 a stale release frees a later claim credential-claim (X5), grant-executor
B37 register replaces an entry with a live token credential-claim (X4)
B38 drop the creative check in #hostDefinition grant-host (mos-1163), grant-executor (mos-1163)
B39 drop the creative check in #hostAccess grant-executor (sage-1908)

B17 may survive. await null yields only to microtasks, and every revoke starts in a macrotask of its own (a socket request, or a test's call), so none can land in that gap. If it survives I'll report it as equivalent under this build, not add a test that fakes a microtask revoke.

Section 10 items not run as row b mutants, and why:

  • The upsert mutant ("let the undo to previous insert a record") and "run past a newer grant.revoked". Row 235 has no undo to previous. A commands change commits the UPDATE and grant.applied in one transaction (sage-1745), so a restart after it leaves the record as it is (G11). The UPDATE can't meet a missing record either: step 1 reads the live record, and nothing awaits between there and the UPDATE. An upsert there would be equivalent.
  • "let the executor act on an abort after step 5 commits". The record and grant.applied commit together, and no abort check follows the commit, so no code path reaches the if (committed) guard with an abort. Removing the guard is equivalent. C1's case is covered by G28 with a later revoke.
  • "treat a missing bus.tester as dev". The tester check can't be reached from a test (step 5 notes).
  • Row a's: the keyscan compare, the writer items, the "not yet enrolled" check, the hosts primary key and host.run reading the business file. Row 234 ran those as A5, A6, A8, A9, A11 and A13-A19.