row 234 (#1740): duckworth-0081 B1-B3 tests (G10, G18, G6 withdraw and arbiter, G20 human check), N2 and N3 fixes, BUILD.md decisions

- broker: #runEnrollment returns its code; public runEnrollment is a trusted entry for G10
- grant-enroll.test: G6 withdraw/arbiter, G10, G20 human check, G18 broker half
- host-run.test: G18 wrong pin refuses host-key-mismatch, nothing runs on the host
- mosaic-keys-write: sudo -l timeout refuses user-sudo (N2)
- host-grant: keyscan comment matches the code (N3)
- BUILD.md: decisions for B1, N1-N5, line numbers, status

agents/gizmoduck/work/queue-234/BUILD.md

@@ -140,35 +140,37 @@ abort after step 5 and a missing `bus.tester` as dev.
 ## Broker functions this row touches (sage-1677)
 
 For row 236 (Megavolt), which rebuilds on this landing. Old lines are `packages/bus/src/broker.mjs` at
-8fede5fc6, new lines are at 9f90d5f4d. The planned list is kept in this file's history (827b89ea6).
+8fede5fc6, new lines are at the last commit that changed this file (they moved by 10 after `runEnrollment`
+went in). The planned list is kept in this file's history (827b89ea6).
 
 Changed:
 - `constructor` (:279): one more option, `hostGrant`, the object `host-grant.mjs` builds.
-- `recordEvent` (old :1243-1251, new :1596): one more refusal. An `action.allowed` whose `body.action` or
+- `recordEvent` (old :1243-1251, new :1606): one more refusal. An `action.allowed` whose `body.action` or
   `body.operation` is a `grant.*` name refuses `reserved-event`.
-- `request()` (old :1362, new :1716): two routes ahead of the sync path, a `decision.raise` with `grant` to
+- `request()` (old :1362, new :1726): two routes ahead of the sync path, a `decision.raise` with `grant` to
   `#raiseGrant` and the `hosts.*` verbs to `#requestHosts`. Both are async.
-- `requestHost` (old :1712-1795, new :2069): the business-file lookups (old :1730-1736) become one
+- `requestHost` (old :1712-1795, new :2079): the business-file lookups (old :1730-1736) become one
   `#hostAccess` call, in the same transaction as `#agent`. `commands` and `entry` come from it.
-- `#dispatch` (old :1796, new :2150) gets a fifth parameter, `enroll`, which only `#raiseGrant` passes.
-  - `decision.raise` (new :2234): `grant` joins the allowed keys, and `grant` without `enroll` (or the
+- `#dispatch` (old :1796, new :2160) gets a fifth parameter, `enroll`, which only `#raiseGrant` passes.
+  - `decision.raise` (new :2244): `grant` joins the allowed keys, and `grant` without `enroll` (or the
     reverse) refuses `invalid-request`. The `host.sudo` block reads `#hostAccess` in place of `b.hosts`
     and the role's `hosts`, and an undefined `target` still refuses `host-unknown`. `validArgv` and
     `validStdin` now run after the access check, so a malformed argv for an ungranted host refuses
     `host-not-granted` first. The enrollment block is appended to the question before `string(question)`,
     so the 4096 cap counts it (M1). The decision's raise body carries `grant: enroll`.
-  - `decision.resolve` (new :2399): when `op` is `resolved`, `#approvedGrant` reads the decision, and a
+  - `decision.resolve` (new :2409): when `op` is `resolved`, `#approvedGrant` reads the decision, and a
     `host-enroll` grant starts `#runEnrollment` from `setImmediate`, after the transaction commits. The
     result adds `grant: 'started'`.
 
-New, all private but one:
+New, all private but two:
 - `#grantEvent` (:657), the writer of guarded events. It runs the secret check, then `writeGrantEvent`.
 - `#approvedGrant` (:667), `#consumeGrant` (:678). `#consumeGrant` sits after `#approvedGrant`, not next
   to `#consumeAuthority` as planned.
 - `#hostRecord` (:685), `#hostAccess` (:692), `#grantNotice` (:708), `#enrollFailed` (:736),
-  `#runEnrollment` (:742), `#openGrantDecisions` (:813).
-- `reconcileGrants` (:823), public, which `runtime.mjs` calls once at start.
-- `#raiseGrant` (:856), `#hostsCaller` (:876), `#requestHosts` (:890), `#hostLinesLeft` (:954).
+  `#runEnrollment` (:750), `#openGrantDecisions` (:823).
+- `runEnrollment` (:745), public, a trusted entry to `#runEnrollment` for G10. No verb reaches it.
+- `reconcileGrants` (:833), public, which `runtime.mjs` calls once at start.
+- `#raiseGrant` (:866), `#hostsCaller` (:886), `#requestHosts` (:900), `#hostLinesLeft` (:964).
 
 Module level, new (:184-230): `GRANT_RUN`, `grantName`, `GRANT_EVIDENCE` (the N1 filter as SQL),
 `registrySha256`, `writeGrantEvent` (exported, the one writer of guarded events, which test fixtures use
@@ -217,10 +219,26 @@ Other `src` files:
   regular, owned by the owner uid and not group or world writable, and are opened without following a
   symlink. Lines starting with `#` are comments. A refusal exits 1 with its code on stderr and in syslog.
   `add` of a tag that is already there exits 0 and writes nothing.
+- **`runEnrollment` is public for G10 (duckworth-0081 B1).** G10 needs a second start for a spent
+  decision, and the resolve path can't give one: a second resolve refuses `decision-closed`
+  (broker.mjs:2416) before the executor runs. So
+  `#runEnrollment` returns its code (`enrolled`, `host-enrolled` or the refusal), and a public
+  `runEnrollment` calls it, as `reconcileGrants` is public. No verb reaches it. The test checks a spent
+  decision gives `decision-consumed`, and an open or agent-resolved one gives `not-authorized`, with
+  nothing written. That second case also reaches the `via` and `by` checks in `#approvedGrant`, so A3 is
+  no longer vacuous (N1).
+- **A `sudo -l` that times out refuses `user-sudo` (N2).** `_sudo` returns an empty answer on
+  `TimeoutExpired`, which isn't the expected line, so `main` logs the refusal and exits 1.
+- **Keyscan comment (N3).** A scan that can't start refuses `host-failed`. One that runs and finds no key
+  returns no lines, and the raise refuses `host-key-mismatch`. The comment in `host-grant.mjs` now says so.
+- **No tag on in-run `grant.failed` (N4).** r6:578 tags only the `interrupted` one, so the code follows it.
+- **Run sheet (N5).** The writer refuses `config-privileged` when the file is missing, so each host needs an
+  empty, root-owned `/etc/mosaic-keys/privileged` (0644) before the first enrollment.
 
-## Status at 9f90d5f4d
+## Status after duckworth-0081
 
-Done: every Split item, 1 to 8. Single files green: grant-enroll 20, host-run 23, hard-waits 13,
+Done: every Split item, 1 to 8, and duckworth-0081 B1 to B3 (G10, G18, the G6 withdraw and arbiter
+cases, G20's human check). Single files green: grant-enroll 24, host-run 24, hard-waits 13,
 host-tools 6, grant-migration 5, project-runtime 3, credentials 5, credential-verbs 2, decision-route 5,
 account-migration 3, tester-writes 8, seat sandbox 10, business hosts 4, business 16, cli hosts 5, cli
 app 30, cli host 42, cli names 2, cli credentials 5.

packages/bus/src/broker.mjs

@@ -738,7 +738,15 @@ export class Broker {
     this.#grantEvent(business, { action: 'grant.failed', grantDecision: decision, host, code, ...extra }, host);
     this.#grantNotice(business, decision, `Enrollment of host ${host} (decision ${decision}) failed: ${code}.`);
   }
-  // r6 E2, started by the human's cli yes. Never throws: the outcome is in the store and the notices.
+  // Trusted host API, as reconcileGrants is, and no verb reaches it: the executor's entry for one
+  // decision. Step 1 checks everything again, so a second start for a spent decision ends
+  // decision-consumed, and an unresolved or agent-resolved one ends not-authorized, with nothing
+  // written (r6 G10).
+  runEnrollment(business, decision) {
+    return this.#runEnrollment(business, decision);
+  }
+  // r6 E2, started by the human's cli yes. Never throws: the outcome is in the store and the notices,
+  // and the return value is its code, 'enrolled' or the refusal.
   async #runEnrollment(business, decision) {
     let grant;
     try {
@@ -757,12 +765,12 @@ export class Broker {
         );
         return g;
       });
-    } catch {
+    } catch (e) {
       // Spent, not approved or no store: nothing was written. The start reconcile ends a resolved,
       // unspent decision.
-      return;
+      return e instanceof BusError ? e.code : 'host-failed';
     }
-    if (!grant) return;
+    if (!grant) return 'host-enrolled';
     const host = grant.host;
     try {
       if (!this.#hostGrant) fail('host-not-configured');
@@ -798,6 +806,7 @@ export class Broker {
             `\nThen run: mosaic hosts check ${host}`,
         );
       });
+      return 'enrolled';
     } catch (e) {
       const code = e instanceof BusError ? e.code : 'host-failed';
       try {
@@ -806,6 +815,7 @@ export class Broker {
       try {
         this.#store.transaction(() => this.#enrollFailed(business, decision, host, code));
       } catch {}
+      return code;
     }
   }
   // r6 X3 and duckworth-0067 N1: credential.mint decisions for `method`, resolved yes by the human via

packages/bus/src/host-grant.mjs

@@ -83,7 +83,9 @@ export function scanLines(stdout) {
 }
 
 // E1: `ssh-keyscan -T 5 -p <port> -t ed25519,ecdsa,rsa <address>` from the stack host. It reads public
-// keys and logs in to nothing. A scan that can't run refuses host-failed.
+// keys and logs in to nothing. A scan that can't start refuses host-failed. One that runs and gets no key,
+// as from a host that doesn't answer, returns no lines, so the raise refuses host-key-mismatch
+// (duckworth-0081 N3).
 export function keyscan(address, port, { bin = 'ssh-keyscan', limitMs = SCAN_MS } = {}) {
   return new Promise((resolve, reject) => {
     const child = spawn(KILL_WITH_PARENT[0], [...KILL_WITH_PARENT.slice(1), bin, '-T', '5', '-p', String(port), '-t', SCAN_TYPES, '--', address], {

packages/bus/tests/grant-enroll.test.mjs

@@ -7,9 +7,10 @@ import { execFileSync } from 'node:child_process';
 import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync, statSync } from 'node:fs';
 import { tmpdir } from 'node:os';
 import { join } from 'node:path';
-import { Broker, writeGrantEvent, registrySha256 } from '../src/broker.mjs';
+import { Broker, BusError, writeGrantEvent, registrySha256 } from '../src/broker.mjs';
 import { Store } from '../src/store.mjs';
 import { hostGrant, fingerprint, grantTag, installerTag } from '../src/host-grant.mjs';
+import { grantDecision } from './host-fixtures.mjs';
 
 const options = [{ key: 'yes', text: 'Yes' }, { key: 'no', text: 'No' }];
 const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
@@ -247,6 +248,43 @@ test('A3, X1: a resolve by the tester or an agent starts nothing', async (t) =>
   assert.equal(f.hostRow(), undefined);
 });
 
+test('G6 (duckworth-0081 B3): a withdraw by the chief or the human, and an arbiter resolve, start nothing', async (t) => {
+  const f = setup(t);
+  for (const who of ['chief', 'human']) {
+    const d = await f.raise();
+    assert.deepEqual(f.call(who, 'decision.withdraw', { id: d.id }), { id: d.id, op: 'withdrawn' });
+  }
+  // A grant raise always routes to the human, so the fixture writes an open one routed to the pm
+  // arbiter, which then resolves it yes through the verb.
+  const d = { id: grantDecision(f.store, { route: 'pm', grant: f.grant(), close: null }) };
+  assert.deepEqual(f.call('pm', 'decision.resolve', { id: d.id, choice: 'yes' }), { id: d.id, op: 'resolved' });
+  await sleep(200);
+  assert.deepEqual(f.grantEvents(), []);
+  assert.equal(f.hostRow(), undefined);
+  assert.deepEqual(f.broker.reconcileGrants(), []);
+  assert.equal(await f.broker.runEnrollment('demo', d.id), 'not-authorized');
+  assert.deepEqual(f.grantEvents(), []);
+});
+
+test('G10 (duckworth-0081 B1): a second start for a spent decision refuses decision-consumed, and an unresolved or agent-resolved one not-authorized', async (t) => {
+  const f = setup(t);
+  const { d } = await approve(f);
+  const before = { events: f.grantEvents(), row: f.hostRow(), messages: f.messages().length };
+  assert.equal(await f.broker.runEnrollment('demo', d.id), 'decision-consumed');
+  assert.deepEqual(f.grantEvents(), before.events);
+  assert.deepEqual(f.hostRow(), before.row);
+  assert.equal(f.messages().length, before.messages);
+  // Unresolved, then resolved by an agent: written by SQL, since the verb refuses the agent first.
+  const open = await f.raise({ host: 'web2' });
+  assert.equal(await f.broker.runEnrollment('demo', open.id), 'not-authorized');
+  f.store.run('INSERT INTO decision_events(decision,at,op,by,choice,via) VALUES(?,?,?,?,?,?)', open.id, new Date().toISOString(), 'resolved', 'chief', 'yes', 'broker');
+  assert.equal(await f.broker.runEnrollment('demo', open.id), 'not-authorized');
+  assert.deepEqual(f.broker.reconcileGrants(), []);
+  assert.deepEqual(f.grantEvents(), before.events);
+  assert.equal(f.hostRow('web2'), undefined);
+  assert.equal(existsSync(f.keyFile('web2')), false);
+});
+
 test('A8, S1: two approved enrollments for one host: the second ends host-enrolled and the pin stays', async (t) => {
   const f = setup(t);
   const d1 = await f.raise();
@@ -349,6 +387,27 @@ test('E4: hosts.check sets the record ready and records grant.host.checked; a no
   assert.deepEqual(status.map((h) => [h.host, h.state, h.hostKeys, h.hostLineLeft]), [['web1', 'ready', f.hostKeys.map(fingerprint), []]]);
 });
 
+test('G20 (duckworth-0081 B3): the human\'s hosts.check marks the host ready, and the reader refuses', async (t) => {
+  const f = setup(t);
+  await approve(f);
+  await refusesWith(f.call('reader', 'hosts.check', { host: 'web1' }), 'read-only');
+  const r = await f.call('human', 'hosts.check', { host: 'web1' });
+  assert.equal(r.state, 'ready');
+  assert.equal(f.hostRow().state, 'ready');
+  const checked = f.grantEvents().filter((e) => e.action === 'grant.host.checked');
+  assert.deepEqual(checked.map((e) => e.by), ['jason']);
+});
+
+test('G18 (duckworth-0081 B2), broker half: a check refused host-key-mismatch leaves the host pending-install with no grant.host.checked', async (t) => {
+  const f = setup(t, { check: () => { throw new BusError('host-key-mismatch'); } });
+  await approve(f);
+  await refusesWith(f.call('chief', 'hosts.check', { host: 'web1' }), 'host-key-mismatch');
+  assert.equal(f.checks.length, 1);
+  assert.equal(f.hostRow().state, 'pending-install');
+  assert.equal(f.hostRow().checked_at, null);
+  assert.deepEqual(f.grantEvents().filter((e) => e.action === 'grant.host.checked'), []);
+});
+
 test('E4: a check whose installer fingerprint differs from the record refuses host-changed', async (t) => {
   const f = setup(t, { check: () => ({ installerFingerprint: 'SHA256:' + 'A'.repeat(43), exit: 0 }) });
   await approve(f);

packages/bus/tests/host-run.test.mjs

@@ -15,6 +15,7 @@ import { fixture } from './spec-fixtures.mjs';
 import { HUMAN_YES, hostGrantRecord, hostRecord } from './host-fixtures.mjs';
 import { Broker, ORGANIZATION_KEYS } from '../src/broker.mjs';
 import { Store } from '../src/store.mjs';
+import { hostGrant } from '../src/host-grant.mjs';
 import { startBroker } from '../src/runtime.mjs';
 import { businessDoc, rolesCopy, systemFor, writeJson } from '../../business/tests/helpers.mjs';
 
@@ -291,6 +292,26 @@ test('T14: a host key not in the registry refuses and nothing runs', async (t) =
   assert.deepEqual(f.argvSeen(), []);
 });
 
+// Row 234 (#1740, r6 G18, duckworth-0081 B2): hosts.check's `list` goes through runHost with the record's
+// pinned keys. A stub `list` on the scratch sshd's PATH records each run.
+test('G18: hosts.check with a wrong pin refuses host-key-mismatch and runs nothing on the host', async (t) => {
+  const f = await hostFixture(t);
+  writeFileSync(join(f.dir, 'fake', 'list'), `#!/bin/sh\necho "$@" >> '${f.dir}/list.log'\n`, { mode: 0o755 });
+  const configDir = join(f.dir, 'cfg');
+  const keyDir = join(configDir, 'secrets', 'hosts', 'demo', 'web1');
+  mkdirSync(keyDir, { recursive: true, mode: 0o700 });
+  for (const d of [join(configDir, 'secrets'), join(configDir, 'secrets', 'hosts'), join(configDir, 'secrets', 'hosts', 'demo'), keyDir]) chmodSync(d, 0o700);
+  writeFileSync(join(keyDir, 'installer'), readFileSync(f.keyFile), { mode: 0o600 });
+  const hg = hostGrant({ configDir, runRoot: f.root });
+  const record = { address: '127.0.0.1', port: f.port, user: 'jarvis', hostKeys: [pubLine(join(f.dir, 'otherhost'))] };
+  await refusesWith(hg.check('demo', 'web1', record), 'host-key-mismatch');
+  assert.equal(existsSync(join(f.dir, 'list.log')), false);
+  // Control: with the right pin the same command reaches the stub.
+  const out = await runHost({ root: f.root, hostId: 'web1', entry: f.entry, key: f.key, timeout: 10, command: 'list jarvis' });
+  assert.equal(out.exit, 0);
+  assert.equal(readFileSync(join(f.dir, 'list.log'), 'utf8'), 'jarvis\n');
+});
+
 test('N3: a remote exit 255 that prints the pin refusal text is the command result, and an ssh failure keeps its message', async (t) => {
   const f = await hostFixture(t, { commands: [['fail255']] });
   const out = await f.call(['fail255']);

tools/host/mosaic-keys-write

@@ -64,14 +64,18 @@ def _groups(user):
 
 
 def _sudo(user):
-    out = subprocess.run(
-        ['/usr/bin/sudo', '-l', '-U', user],
-        env={'LC_ALL': 'C', 'PATH': '/usr/bin:/bin'},
-        stdin=subprocess.DEVNULL,
-        stdout=subprocess.PIPE,
-        stderr=subprocess.STDOUT,
-        timeout=10,
-    )
+    try:
+        out = subprocess.run(
+            ['/usr/bin/sudo', '-l', '-U', user],
+            env={'LC_ALL': 'C', 'PATH': '/usr/bin:/bin'},
+            stdin=subprocess.DEVNULL,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.STDOUT,
+            timeout=10,
+        )
+    except subprocess.TimeoutExpired:
+        # No answer is not the expected line, so the caller refuses user-sudo and logs it (duckworth-0081 N2).
+        return ''
     return out.stdout.decode('utf-8', 'replace')