13 KiB
Row 111 (#1605) round 1: bus host start, the steps after the broker's reply and the child error event
Brief: docs/plans/2026-10-13_landing-follow-ups.md, section "Bus host start: release after the broker's reply, and the child error event" (blob 19c939e10 at 02fe2d9d5).
BASE: 02fe2d9d55. Candidate: build.patch against BASE. candidate-manifest.sha256 lists the 2 files, and applying the patch to a fresh worktree at BASE reproduces both digests. No commit after BASE touches either path. Per the brief, packages/cli/README.md is unchanged; row 110 owns it next.
What changed
All in packages/cli/src/host.mjs, startHost and its firstReply helper.
- One failure path after the broker's reply. The reader lookup, the notifier's fork, its start send, the wait for its reply, a refusal,
rmSyncof an old host.json andwriteHostStatenow all run in onetry. Itscatchsends the notifier SIGTERM if one was forked and waits up to 5 s for it, sends the brokercloseif the channel is still up, waits up toCLOSE_TIMEOUT_MSfor the broker, callsrelease()and rethrows. That is the path the notifier's no-reply case already took. At BASE, a throw from the notifier fork, the start send, the reader lookup orwriteHostStateleft host.lock and a live broker. - A missing reader. When the broker's reply has no reader for
business, the start refuses withbroker has no reader for business <id>, exit 3. At BASE that was a TypeError onreader.cap. The check runs with and without a notifier. Without one, BASE started a host anyway;mosaic bus startalways boots the broker for the business it names, so the only change in practice is a refusal where BASE threw a TypeError. - The notifier's wait starts after its send, as the broker's has since row 106. At BASE the wait started first, so a send that threw would have left a wait whose later rejection nobody handled.
- A notifier that refuses now gets SIGTERM before the wait, like every other failure. It has already called
stop(3)or is about to,stopruns once, and SIGTERM to a child that has exited does nothing, so the refusal and its exit code are unchanged. - The child
errorevent. Node emitserroron a child when a send with no callback fails, or a kill fails with anything but ESRCH. With no listener, the emitter throws, the process ends, and host.lock and host.json stay.startHostnow puts a listener on each child right after its fork:- During the start,
firstReplyalso rejects onerror, with<broker|notifier> failed before it replied: <message>, exit 1, so the start fails at once instead of at the reply timeout. Anerrorat any other time during the start is kept, and the sharedtrythrows it as<name> failed: <message>just before host.json is written. A brokererrorwhile the notifier starts is the case that reaches this check. Anerrorduring either failure path is kept and dropped, since the start is failing already. - Once the host is up, an
erroron the broker breaks the channel throughbreakChannel("broker error: <message>"). That rejects the request waiting on the broker, if any, logs and callsclose(1). Anerroron the notifier logsnotifier error (<message>); stopping the hostand callsclose(1), as a notifier that dies does. Either waydoneresolves with 1 and close() removes host.json and host.lock.
- During the start,
An error after a successful start
The brief asks what an error after a successful start does and why. At BASE it ended the process with host.lock and host.json left behind. request() sends to the broker with no callback, so a failed send there emits error with nothing listening. The lock was left held, so the brief puts the change in scope. I chose to stop the host with exit 1. The unit restarts it, and that is what a child's death already does (watchChildren). For the broker the error goes through breakChannel: a request whose message never reached the broker would otherwise wait for its timeout and then break the channel anyway. A kill that fails on a running host means a child we can't signal, and stopping the host is the fail-closed answer there too.
The up flag switches the listener from the start behaviour to the running one. It is set in the same synchronous run as host.json's write, after breakChannel, stopping and close exist, so no event can reach the listener in between.
Tests
packages/cli/tests/host.test.mjs: 31 at BASE, 41 in the candidate. The 10 new ones:
| Test | What it checks |
|---|---|
a notifier fork that throws stops the broker ... (loop, fork and send) |
the throw's own error; the broker has ended; with send, the notifier has ended too; no host.lock, host.json or writer.lock |
a boot reply with no reader for the business refuses with exit 3 naming it, ... (loop, with and without a notifier) |
exit 3 and the message; no notifier forked; the broker has ended; no host.lock, host.json or writer.lock |
a host.json that can't be written stops both children and leaves no host.lock |
EEXIST from writeHostState; both children ended; no host.lock, host.json or writer.lock |
| `an error event on the <broker | notifier> before it replies fails the start at once, ...` (loop) |
an error event on the broker while the notifier starts fails the start once the notifier replies |
broker failed: write EPIPE; both children ended; no host.lock or host.json |
| `an error event on the <broker | notifier> once the host is up stops the host with exit 1, ...` (loop) |
How the tests make each failure:
- Fork and send throws use
throwOn. It now takesnth, the fork it targets, and leaves other children alone; the row 106 broker tests pass the default 0. A fork that throws returns no child, sostartHostcan't stop that notifier;throwOnkills it after the test, and the fork test checks only the broker. - The no-reader case starts a host for business
otheron a boot config foracme. writeHostStatewrites throughhost.json.<pid>.tmpwith thewxflag. The test makes that file first, so the write throws EEXIST in the test's own process.- The
errorbefore a reply usesfailSends: the boot or start send fails with an EPIPE error, delivered as anerrorevent on the next tick because the send has no callback. The child is left running and never gets the message, so only theerrorcan end the wait in time. - The kept broker
errorand the errors once up are emitted on the child directly, on the next tick in the first case. forksSeenrecords every child forked during the test, so a test can check each one has ended. It kills them all after the test.
Mutants
mutants.py and gate.sh are the scripts I ran, copied here; they point at my scratch worktree, so change WT/G to rerun them. mutants.py runs each mutant against the named tests only, through --test-name-pattern, and counts a pattern that matches no test as a fault. n00 puts BASE's host.mjs, read from the worktree's HEAD, under the new tests.
Final run 23:32:52 to 23:33:46Z on 2026-10-13. Each file is restored from its .orig, and the candidate's diff hashed 2ef465444b6936ca (first 16 hex) before and after. Every run matched the tests it names: 1 each, 2 for the two loops that share a prefix.
| Id | Mutant | Tests | Result and how it fails |
|---|---|---|---|
| n00 | BASE's host.mjs under the new tests | all 9 patterns | killed, all 9: a child still running, Missing expected rejection, or the reader.cap TypeError |
| n01 | firstReply doesn't listen for error |
error before reply, broker and notifier | killed: both hit the 8 s test timeout |
| n02 | no error listener on the broker |
kept broker error; broker error once up | killed: the error throws with nothing listening |
| n03 | no error listener on the notifier |
notifier error once up | killed: the error throws with nothing listening |
| n04 | a broker error once up is dropped |
broker error once up | killed: 15 s test timeout, the host never stops |
| n05 | a notifier error once up only logs |
notifier error once up | killed: 15 s test timeout, the host never stops |
| n06 | no reader check | no reader, both | killed: Missing expected rejection without a notifier, the reader.cap TypeError with one |
| n07 | the notifier fork runs before the shared try |
notifier fork throws | killed: the broker has not ended |
| n08 | the notifier's wait starts before its send | notifier send throws | killed: the abandoned wait rejects with notifier exited (null) before it replied and nothing handles it |
| n09 | host.json is written after the shared try |
host.json write | killed: the children have not ended |
| n10 | a kept child error is not thrown |
kept broker error | killed: Missing expected rejection |
| n11 | the failure path leaves the notifier running | host.json write; notifier send throws | killed: the notifier has not ended |
| n12 | the failure path doesn't wait for the broker | no reader, both; host.json write | killed: the broker has not ended |
| n13 | the failure path keeps the lock | no reader, both; host.json write; notifier fork throws | killed: host.lock is still there |
Two earlier runs did not count, and their logs are here as mutants-run-hung.log and mutants-run-count.log.
- The first hung on n00. BASE leaves the broker and notifier running after a failed start, and their IPC channels kept the test file from exiting, so the run sat at mutants.py's 600 s timeout. I stopped it; mutants.py put the candidate's host.mjs back.
forksSeennow kills every child it saw after the test, asthrowOnandfailSendsalready did, so a start that leaves children running fails the test instead of hanging it. The host tests reran at 41 passed, 0 failed. - The second reported n01, n04 and n05 as faults with no tests run. Each of those tests had hit its timeout, which node counts as cancelled, not failed, and mutants.py counted only passes and fails. It now counts cancelled tests too. That run had every other mutant killed, and n00 killed on all 9 patterns.
Gate
gate.sh, on BASE 02fe2d9d5 with the final candidate, 23:33:50 to 23:39:29Z on 2026-10-13, DOCKER_HOST unset, Docker 29.7.2.
| Step | Result | Time (UTC) |
|---|---|---|
| pkg-bus | 128 passed, 0 failed | 23:33:50-23:33:53 |
| pkg-business | 67 passed, 0 failed | 23:33:53-23:33:55 |
| pkg-cli | 178 passed, 0 failed | 23:33:55-23:34:16 |
| pkg-control-board | 124 passed, 0 failed | 23:34:16-23:34:16 |
| pkg-conversation | 182 passed, 0 failed | 23:34:16-23:35:05 |
| pkg-discord | 178 passed, 0 failed | 23:35:05-23:35:08 |
| pkg-harness | 73 passed, 0 failed | 23:35:08-23:35:18 |
| pkg-ledger | 78 passed, 0 failed | 23:35:18-23:35:38 |
| pkg-mosaic | 69 passed, 0 failed | 23:35:38-23:35:40 |
| pkg-queue | 156 passed, 0 failed | 23:35:40-23:36:11 |
| pkg-runs | 43 passed, 0 failed | 23:36:11-23:36:12 |
| pkg-seat | 119 passed, 0 failed | 23:36:12-23:36:17 |
| pkg-tasks | 92 passed, 0 failed | 23:36:17-23:36:22 |
| pkg-webui | 108 passed, 0 failed | 23:36:22-23:37:18 |
| script-auth | 15 passed, 0 failed | 23:37:18-23:37:19 |
| script-conductor | 17 passed, 0 failed | 23:37:19-23:37:25 |
| script-config | 24 passed, 0 failed | 23:37:25-23:37:26 |
| script-discord | 66 passed, 0 failed | 23:37:26-23:37:34 |
| script-extension-package | 18 passed, 0 failed | 23:37:34-23:37:35 |
| script-foundation | 44 passed, 0 failed | 23:37:35-23:38:15 |
| script-queue | 28 passed, 0 failed (node 156 passed) | 23:38:15-23:38:48 |
| script-release | 14 passed, 0 failed | 23:38:48-23:38:58 |
| script-task | 98 passed, 0 failed | 23:38:58-23:39:29 |
The gate printed GATE_PASSED at 23:39:29Z. The Docker steps left a wt_default network, created at 23:38:48Z with 0 containers; I removed it. An earlier gate run on the candidate before the forksSeen change also passed (23:13:16 to 23:19:01Z, gate-run-r0.log); its network, created at 23:18:19Z with 0 containers, is removed too.
Choices and limits
- The broker's own refusal (
ready.oknot true) keeps its BASE path. It runs before the sharedtry, and the broker exits on its own after refusing. - The shared
catchdoes not remove host.json.writeHostStateis the last step in thetry, so a host.json this start wrote is never followed by a throw. ThermSyncbefore it removes a stale one only, as at BASE. - If
renameSyncinwriteHostStatethrows,host.json.<pid>.tmpstays. The start still fails and gives the lock back.writeHostStateis outside this row's scope, and the name is per pid. - The third-racer window,
stopHost,hostStatus, host.json's format and the broker are untouched, as the brief says.
Going live (decision 109)
The host change is live only with a new copy, a re-pin and a bus restart, all under Mos's go. Until then the live bus, Console and CLI run host.mjs from the pinned copy at 465bbdfbb. The tests change nothing that runs.