Compare commits

..
Author SHA1 Message Date
jason.woltje f268f4de49 row224: record whole-suite passes and killed retry mutant 2026-10-24 13:43:47 -05:00
jason.woltje e4c0c8931e row224: supply required reference in creative tool fixture 2026-10-24 13:41:58 -05:00
jason.woltje 63827cc0bd row224: record CLI and bus passes and harness fixture red 2026-10-24 13:40:57 -05:00
jason.woltje 8f1b045f40 row224: await citation hydration in new launch test 2026-10-24 13:39:55 -05:00
jason.woltje b98b9e64a0 row224: record new CLI test failure and held sequence 2026-10-24 13:37:44 -05:00
jason.woltje 6160b49810 row224: supply chief Gitea reference in new launch fixture 2026-10-24 13:31:19 -05:00
jason.woltje 5c5886d2d8 row224: record local install and single CLI run failure 2026-10-24 13:21:02 -05:00
jason.woltje ed985c0442 row224: trim whitespace in plain-text failure receipt 2026-10-24 13:15:54 -05:00
jason.woltje 532938beeb row224: fix new fixture setup and record old-test stop 2026-10-24 13:15:45 -05:00
jason.woltje a1a7357457 row224: record row266 rebase and whole-file gate list (unrun) 2026-10-24 13:02:16 -05:00
jason.woltje c509dcb6ee row224: record static rebase and pending gate (unrun) 2026-10-24 13:02:00 -05:00
jason.woltje 82d6cc62c6 row224: pace failed inline sync attempts (unrun) 2026-10-24 13:02:00 -05:00
jason.woltje 212aef6161 row224: draft broker library lookups and typed tools (unrun) 2026-10-24 13:02:00 -05:00
jason.woltje 3e5041b43b Row 266: record owner review move and missing request credential 2026-10-24 12:11:42 -05:00
jason.woltje b5fb6280df Row 266: record full gate and sixteen isolated mutant kills 2026-10-24 12:11:42 -05:00
jason.woltje e1092e673e Row 266: assert allowed room plugin dispatch does not reject 2026-10-24 12:11:42 -05:00
jason.woltje abc11152a0 Row 266: add Console room timeline and member completion 2026-10-24 12:11:42 -05:00
jason.woltje 73692e1bb6 morgana: row 257 slot result on 1261fa2d8 (sage-4159): three files whole 15/29/1 pass, supervisor suite 45/45, mutants M1-M8 all killed by assertion, tree restored; BUILD.md and evidence only 2026-10-24 12:02:34 -05:00
jason.woltje fc5f282902 morgana: row 257 r2 (filbert r1 N1, N2): Object.hasOwn unknown-key refusal with inherited-name cases; critical memory edge on a 20 GiB limit (M8) 2026-10-24 12:02:34 -05:00
jason.woltje 045dd758a3 morgana: row 257 BUILD.md records sage-2202 rulings (PSI latch on first seen, PSI count across resources) 2026-10-24 12:02:34 -05:00
jason.woltje 2d7ebe012d morgana: row 257 build (Q198 S1, #1762): @mosaic/supervisor pressure parsers and pure D4 rules evaluator, tests and mutant plan; tests wait for Sage's slot 2026-10-24 12:02:34 -05:00
21 changed files with 2759 additions and 5 deletions

No files matched your search

+742
View File
@@ -0,0 +1,742 @@
# Row 224 — C2-05b broker lookups and typed tools (#1725)
Owner: Della; reviewer: Dewey. Assignment sage-2795; scope extension sage-2804.
Base: `4f3fbc25a` (Researcher C2-05a). Branch: `della/row224`.
Brief: C2-05b in `docs/plans/2026-10-20_case2-rows.md` at `aca16404c`
and `51cc0445c`; design section 3, 5.3 M3 and 7.2; C2-05a BUILD rulings.
## Execution boundary and progress
Code/docs and local Git only while row 253 holds the gate. No Node (including
syntax checks), HTTPS Git, queue claim, test or push until Sage's CLEAR.
Tests are written first; their RED/GREEN runs are deferred, not claimed.
Queue next is deferred too: its executable uses Node. Sage assigned this row
explicitly. After 253 and 216 land, rebase under a new branch name.
- Pre-flight: query.mjs consumes `{repo,commit,mapping}`; broker cannot import
Git/filesystem library code because it is in the sandbox import closure.
Runtime injects the host lookup adapter; existing human file/status/sync stay.
- Pre-flight: resolved has no project list and a library is never a checkout.
Bundle needs an explicit library flag from launcher business configuration.
- M3 reading: exact creative base is five lookups plus send_message; retain
explicitly granted broker web_search/web_fetch for research/companion under
landed C2-08 and sage-2804. No generic spec, task, evidence or decision tools. Residual (mos-1308): those
reads lack broker action checks; their typed-tool absence is the only fence
for creative roles. M1/M2 deny bash; broker-side refusal is outside this row.
Mos confirmed this interpretation in mos-1308; design wording fix belongs to Gosalyn. Other definitions retain their current tools.
- Multi-library businesses use an optional project ID; omit it only with one
configured library. Universe is required for all five tools, including get.
- Missing library uses the brief's `library-not-configured`; argument errors
reuse `invalid-request` and C2-05a query codes. No new action or authority.
## Manifest (draft code, unrun)
- bus/src/broker.mjs: five lookup verbs, strict args, holder/business rechecks.
- bus/src/runtime.mjs: host lookup adapter, stale-before-read sync, served pin.
- harness/src/tools.mjs: five schemas; creative exact-set filtering.
- harness/src/bundle.mjs: library flag forwarded to typedTools.
- cli/src/launcher.mjs: one flag derived from trusted business projects.
- bus/tests/library-lookups.test.mjs: real query/broker/socket/runtime cases.
- harness/tests/library-tools.test.mjs: schemas, role snapshots, bundle parity.
- cli/tests/thread-launch.test.mjs: flag reaches a real launched bundle with no library checkout.
- agents/della/work/queue-224/BUILD.md: manifest, rulings and deferred verification ledger.
## Verification ledger
No test or syntax-check run yet. Acceptance requires full bus/cli/harness suites
and every scripts/test-*.sh on Sage's gate; focused files and affected CLI/
seat import-closure tests precede that only in an authorized CLEAR.
## Coordination
Huey272 reports no runtime/bundle/launcher overlap. Rocko219 changes launcher
THREAD_DEFINITIONS checks only, separate from our buildBundle flag argument.
The launcher argument is required: bundle cannot derive library presence from
checkout IDs (a library has no agent clone). No role, authority or config schema
changes; library project entries are already validated and protected by M9.
## Mount and pin impact (inspection only)
No new imports in broker.mjs or tools.mjs; no RUNNER_FILES or sandbox mount
addition. runtime.mjs is host-only and already imports library index.mjs.
Bundle forwards only a boolean, never a root/mapping/slot/credential. The
existing import-closure tests are bus/tests/project-runtime.test.mjs and
seat/tests/sandbox.test.mjs. Grep found no literal sha256 pin for runtime.mjs
or bundle.mjs; bundle tests derive current source digests through SOURCES.
Library sources remain in the existing whole-directory release copy.
## Runs to schedule only after CLEAR
Set TMPDIR=/mnt/storage/scratch/tmp. Focused new bus/harness files first,
then existing library.test.mjs, library-sync.test.mjs, project-runtime.test.mjs,
harness creative/tools/spec-tools/bundle files and affected CLI launch tests.
Follow with the required whole bus/cli/harness suites and Sage's full shell gate.
No timing claim yet: use the repository/Sage per-file limits and record every
command, outcome, elapsed time and timeout before acceptance. No focused skip
or pattern run substitutes for whole-file gate evidence.
## Read-through findings and residuals
- Library get requires universe although the older design table omitted it;
section 3 isolation rule and C2-05a's accepted API are binding.
- Query type enum matches mapping.TYPES (five values); manuscript is a distinct
lookup and not a find type. Unknown host-path/commit/business/accept fields
are refused; explicit project is restricted to this business's library list.
- Async reads recheck the live holder and current project revision. A stale
read attempts sync, then takes the new served snapshot; a returned fetch
failure/divergence keeps serving the old pin. Busy sync uses the current pin.
Unreadable state, no served commit or unsafe configuration refuse. Query
and sync never hold a SQLite transaction over await.
- Processing halt refuses new agent lookup work and rechecks before returning
async results. Existing human library file/status/sync behavior is unchanged.
- Timer sync and reads still use C2-03a's 60-second contract. No timer tuning or
resource-limit schema is introduced by this row. Q222 creative reply settings
are in C2-06; no reply guard or writing capability is implemented here.
- The generic tool socket's existing 60-second client timeout can precede the
server's 120-second Git timeout. This row does not change the generic timeout;
a timed-out lookup is read-only and retryable, never a write/outcome claim.
- All tests are authored but unrun. No RED, GREEN, syntax, runtime performance,
sandbox, broker/harness suite, shell gate, claim or push evidence exists yet.
## Candidate review checklist (read-through only)
- [x] Five verbs mapped to real C2-05a methods, all require universe.
- [x] Default sole library or explicit owned project; no caller repo/commit.
- [x] Human-only file/status/sync gates kept; lookup holder rechecked.
- [x] Async socket dispatch includes lookup verbs through LIBRARY_VERBS.
- [x] Stale/unknown fetch syncs first; results use current immutable served pin.
- [x] Six/eight creative names per mos-1308; other roles retain their tools.
- [x] Launcher -> bundle -> typedTools passes presence without a library clone.
- [x] No imports added to sandbox source closure; no mount change.
- [ ] Execute new tests, RED characterization, full files/suites after CLEAR.
- [ ] Rebase after rows 253/216 land, under a new branch name.
- [ ] Claim/push only in Sage-authorized gap; Dewey review and Sage gate.
Candidate preservation commits are explicitly unrun under sage-2795; they are
not completion, green-suite or landing claims. No dependencies installed.
Correction: the first committed-range diff --check found a blank line at EOF
in the new library-tools test, missed while it was untracked. Removed in a
follow-up local commit; the complete range must be checked again. No tests ran.
## Shared-log receipts for Sage (sage-2815)
The shared logs are unchanged from the base in this row. Sage appends these
receipts in the landing sweep; they are preserved here, not in the manifest.
For `BUILD-LOG.md`:
```text
2026-10-22T16:36:30Z | Della row224 C2-05b #1725 preparation | Before: C2-05a queries exist at 4f3fbc25a but no broker lookup verbs or typed tools. Scope: broker/tools plus Sage-authorized runtime/bundle integration, tests and docs. Existing human-only library reads retained; five agent queries require a live holder and trusted configured source. Code/docs only under row253 gate; no Node/test/push/claim. Runtime ownership coordinated with Rocko219/Huey272.
```
For `docs/SESSIONS.md`:
```text
2026-10-22T16:36:30Z | Della (sage-2795 / sage-2804, Q220) | Row 224 C2-05b #1725, della/row224 base 4f3fbc25a | Read both pinned briefs and C2-05a source; preparing broker/typed tools and authorized host runtime/bundle seams. No queue command, Node, HTTPS Git, claim, test or push during Gander gate. Dewey reviewer; Rocko/Huey runtime coordination sent. Tests and integration verification deferred to CLEAR.
```
Local-only draft pins 19c0a7adf/7cf39c390 are superseded by the
log-free candidate. Saved as local branch della/row224-draft-with-logs before
rebuilding the unpublished candidate commit; no remote history changed.
## Dewey dewey-0852 and Sage sage-2819 — focused corrections
B1: inline freshness uses max(lastFetch, lastError.at), not lastFetch alone.
An error within 60 seconds counts as a recent attempt, so later lookups serve
the old served pin without another fetch. Keep inline sync: the brief requires
it and businesses without timed libraries have no sync timer. The extended
failed-fetch test makes reads at +1, +59,999 and +60,000 ms (one attempt total),
then +60,001 ms (second attempt). All reads must return the old commit.
Required mutant M-B1: remove lastError.at from recentAttempt. The repeated
reads must fail the attempted-count assertion. This mutant has NOT been run.
Residual (Sage will defer): the first inline fetch in each outage window can
still exceed the tool timeout (60 seconds, or the runner broker client’s
30 seconds). Bounding that fetch is out of scope for 224. Subsequent reads
are paced by persisted lastError.at. No claim of first-read outage latency.
N1: whole bus, cli and harness suites are required, plus the shell gate.
If the new thread-launch integration test skips due to sandboxProblem(),
name the skip; a skip is not launcher integration proof.
N2: cli/tests/thread-launch.test.mjs now depends on library/tests/helpers.mjs
mappingEntry() and library/tests/fixtures/mapping.json. Changes to that fixture
must rerun the CLI launch file.
N3: harness/tests/library-tools.test.mjs compares the type enum to mapping.mjs
TYPES, so adding a mapped type cannot silently leave the typed tool behind.
These edits are local code/docs only. No Node, RED/GREEN run, mutant, claim,
HTTPS Git or push. New local pin goes to Dewey/Mos/Sage for focused review.
## sage-4163 — row224-r2 static rebase, 2026-10-24
New isolated worktree: `/mnt/storage/scratch/della-row224-r2`.
Branch `della/row224-r2`, base refactor `487500a16f4ab5902c318509fdec6acc56a543ca`.
Only the two row-owned commits were cherry-picked, in order:
`38e322ea0` -> `327073400`; `23816cf28` -> `e138b2c55`.
No old prerequisite commits were copied into this series. Rows 216 and 217
are now upstream. Row 266 remains a landing dependency: rebase on its landing
when Sage announces it. No broker hunk touches room.read or its hydration.
One content conflict: the footer of cli/tests/thread-launch.test.mjs. Kept
both upstream C2-09a test bodies and every assertion, followed by our unchanged
C2-05b launched-bundle test. No test was removed or weakened. Range-diff shows
only that context relocation and the upstream sandbox workspace expression
beside launcher's added library flag. The B1 commit is patch-identical.
B1 answer: e138b2c55 retains max(lastFetch, lastError.at), the repeated-read
regression and M-B1 specification. Dewey already checked the original fix at
23816cf28 in dewey-0853; it still needs executed mutant and gate evidence.
N2 dependency also includes harness/tests/library-tools.test.mjs importing
library/src/mapping.mjs TYPES; changes there must rerun that harness file.
Static checks: complete-range git diff --check clean; nine manifest paths.
Scanned packages/*/tests for every changed basename alongside literal 64-hex
hashes (three lines on either side): no matching hash pins. Broker and tools
add no imports; no RUNNER_FILES or sandbox mount changes.
No Node, syntax check, test, mutant, install or build was run. The historical
verification ledger above records the original preparation, not current gate
permission. Under sage-4163 the required gate is Dewey's manifest approval,
whole bus and harness suites, plus every scripts/test-*.sh on Sage's gate;
record every skip. Tests and mutants wait for Sage's explicit CLEAR. Static
Git and pushing this new branch are authorized now. Shared logs remain
unchanged; Sage owns their append-only landing receipts.
## sage-4181 — row266 landing rebase, 2026-10-24
Rebased the complete r2 series onto refactor
`3e5041b43b36d7f1c5ba7ce2f14b3634fe0955db` under new branch
`della/row224-r3`; r2 is preserved, no force-push. No conflicting files and
no manual conflict resolutions. Range-diff: all three prior commits equal
(212aef616, 82d6cc62c, c509dcb6e). B1 remains patch-identical.
Complete-range diff --check is clean. No test, syntax or mutant run.
Own touched test files, each to run whole in Sage's slot:
- packages/bus/tests/library-lookups.test.mjs
- packages/cli/tests/thread-launch.test.mjs
- packages/harness/tests/library-tools.test.mjs
Tests changed by the new upstream base, also named for the rebase-rule gate:
- packages/bus/tests/room-check.test.mjs
- packages/plugins/tests/room-channel.test.mjs
- packages/supervisor/tests/pressure.test.mjs
- packages/supervisor/tests/purity.test.mjs
- packages/supervisor/tests/rules.test.mjs
- packages/webui/tests/room-browser.test.mjs
- packages/webui/tests/room-http.test.mjs
None of those seven upstream test files has a row224-owned diff. Full bus and
harness suites plus every scripts/test-*.sh remain the row gate; the listed
CLI and upstream files require whole-file runs under sage-4181 too. Dewey's
delta review follows Sage's run. No CLEAR is inferred from the rebase.
## sage-4192 CLEAR — run receipt and old-test stop, 2026-10-24
Candidate named by CLEAR: a1a7357457a31d219b99effa9b312ec78d06f060.
Fresh mktemp TMPDIR: /mnt/storage/scratch/tmp/della-224-4192.r3L7cj.
Each command used nice -n 19 and timeout -k 10 300 node --test FILE.
Inherited OS niceness was -4, so nice initially yielded 15; corrected the
current runner tree to absolute 19 during the CLI run. Future runs must set
absolute 19 before spawning. No other seat's processes were changed.
First whole bus/tests/library-lookups.test.mjs run: 11 tests, 8 pass, 3 fail,
0 skipped, duration 1.173 s (exit 1). All three are this row's new tests.
Root causes: Broker clones businesses, so mutating the original fixture did
not publish the second project or the delisting; runtime requires the
selected harness to be enabled. Fixed only new fixture setup: reloadBusiness
after each of the two changes and plugins.harness ['pi'] in runtime fixture.
All assertions and production code are unchanged. Whole-file rerun: 11 pass,
0 fail/cancelled/skipped, 1.322 s (exit 0). B1 regression included and passed;
M-B1 has not been executed.
Next whole cli/tests/thread-launch.test.mjs reported OLD-test reds at :167,
:216 and :241. On observing them stopped only this run's process tree; no
rerun or next file was started. Interrupted-file summary: 0 pass, 3 fail,
1 cancelled, 0 skipped, 62.988 s. Returned slot to Sage in della-0041 under
mos-1592. No old test changed. No in-review move: the third own file, all
seven new-base files, both whole suites and mutant evidence remain pending.
No shell/full landing gate was run.
Exact completed run receipts:
```text
{"name": "packages/bus/tests/library-lookups.test.mjs", "command": ["nice", "-n", "19", "timeout", "-k", "10", "300", "node", "--test", "packages/bus/tests/library-lookups.test.mjs"], "exit": 1, "seconds": 1.173, "TMPDIR": "/mnt/storage/scratch/tmp/della-224-4192.r3L7cj"}
{"name": "packages/bus/tests/library-lookups.test.mjs", "command": ["nice", "-n", "19", "timeout", "-k", "10", "300", "node", "--test", "packages/bus/tests/library-lookups.test.mjs"], "exit": 0, "seconds": 1.322, "TMPDIR": "/mnt/storage/scratch/tmp/della-224-4192.r3L7cj"}
```
Old-test failure output (interrupted whole file, not a whole-file pass):
```text
✖ REQ-THREAD-2 REQ-LAUNCH-1 REQ-MSG-1 a thread message starts the thread's session, which stops when idle; its history survives the stop and a restart and no other thread sees it (20120.78425ms)
✖ REQ-THREAD-2 REQ-LAUNCH-1 a running thread counts against its model's limit and an idle-stopped one doesn't (185.485131ms)
✖ REQ-THREAD-2 REQ-LAUNCH-1 a turn longer than the idle time isn't stopped, and the thread idles out after it (30082.743341ms)
Interrupted while running:
⚠ packages/cli/tests/thread-launch.test.mjs (packages/cli/tests/thread-launch.test.mjs:1:1)
✖ packages/cli/tests/thread-launch.test.mjs (62982.9717ms)
ℹ tests 4
ℹ suites 0
ℹ pass 0
ℹ fail 3
ℹ cancelled 1
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 62987.534861
✖ failing tests:
test at packages/cli/tests/thread-launch.test.mjs:167:1
✖ REQ-THREAD-2 REQ-LAUNCH-1 REQ-MSG-1 a thread message starts the thread's session, which stops when idle; its history survives the stop and a restart and no other thread sees it (20120.78425ms)
Error: timed out waiting
at until (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:48:9)
at async ask (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:153:17)
at async TestContext.<anonymous> (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:179:17)
at async Test.run (node:internal/test_runner/test:1409:7)
at async startSubtestAfterBootstrap (node:internal/test_runner/harness:387:3)
test at packages/cli/tests/thread-launch.test.mjs:216:1
✖ REQ-THREAD-2 REQ-LAUNCH-1 a running thread counts against its model's limit and an idle-stopped one doesn't (185.485131ms)
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected
+ 'harness-binary-missing'
- 'capacity-full'
at TestContext.<anonymous> (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:227:10)
at async Test.run (node:internal/test_runner/test:1409:7)
at async Test.processPendingSubtests (node:internal/test_runner/test:974:7) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: 'harness-binary-missing',
expected: 'capacity-full',
operator: 'strictEqual',
diff: 'simple'
}
test at packages/cli/tests/thread-launch.test.mjs:241:1
✖ REQ-THREAD-2 REQ-LAUNCH-1 a turn longer than the idle time isn't stopped, and the thread idles out after it (30082.743341ms)
Error: timed out waiting
at until (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:48:9)
at async TestContext.<anonymous> (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:251:3)
at async Test.run (node:internal/test_runner/test:1409:7)
at async Test.processPendingSubtests (node:internal/test_runner/test:974:7)
test at packages/cli/tests/thread-launch.test.mjs:1:1
✖ packages/cli/tests/thread-launch.test.mjs (62982.9717ms)
'Promise resolution is still pending but the event loop has already resolved'
```
Receipt correction: copied test output contained two whitespace-only lines;
complete-range diff-check caught them. Trimmed those log lines in this
follow-up; no test/code or failure text changed, and no test was rerun.
## sage-4195 / mos-1595 — pre-install evidence
No canonical node_modules link made; that suggestion was refused under
mos-1575. Before installing anything in this worktree, ran:
```text
$ ls -l node_modules/@earendil-works/pi-coding-agent/dist/bundle/cli.js
ls: cannot access 'node_modules/@earendil-works/pi-coding-agent/dist/bundle/cli.js': No such file or directory
rc=2
```
The file is missing, so the authorized local npm ci follows. This is an
environment finding; prior existing-test reds do not establish a row-code
regression. No old-test edit or rerun preceded this CLEAR.
## sage-4195 — install and single-run STOP receipt
```text
$ [ -L node_modules ] && rm node_modules; timeout -k 10 900 npm ci --ignore-scripts --no-audit --no-fund
npm warn deprecated [email protected]: Use your platform's native DOMException instead
added 128 packages in 2s
npm_ci_rc=0
$ readlink -f node_modules/.bin/pi
/mnt/storage/scratch/della-row224-r2/node_modules/@earendil-works/pi-coding-agent/dist/bundle/cli.js
```
One authorized whole-file invocation, no retry or test edit. Absolute runner
and timeout niceness 19 before launch; observed Node parent/worker niceness
16 during execution (host ananicy rule), recorded without stopping for nice.
Fresh TMPDIR and command in receipt below. Stopped test process tree upon
observing existing-test reds. Interrupted summary: 0 pass, 2 fail, 1 cancelled,
0 skipped; exit 1, 40.056 s. This is not a whole-file pass. The capacity test
now receives launch-failed, rather than harness-binary-missing. Root cause of
that launch failure is not established. No assertion/production changes.
Slot returned to Sage in della-0044 for escalation to Mos; all other files,
suites and M-B1 remain held pending their own CLEAR.
```text
{
"command": [
"timeout",
"-k",
"10",
"900",
"node",
"--test",
"packages/cli/tests/thread-launch.test.mjs"
],
"TMPDIR": "/mnt/storage/scratch/tmp/della-224-4195._yq176su",
"exit": 1,
"seconds": 40.056,
"start_runner_nice": 19,
"observed_timeout_nice": 19
}
```
Failure output:
```text
✖ REQ-THREAD-2 REQ-LAUNCH-1 REQ-MSG-1 a thread message starts the thread's session, which stops when idle; its history survives the stop and a restart and no other thread sees it (20109.08171ms)
✖ REQ-THREAD-2 REQ-LAUNCH-1 a running thread counts against its model's limit and an idle-stopped one doesn't (330.738356ms)
Interrupted while running:
⚠ packages/cli/tests/thread-launch.test.mjs (packages/cli/tests/thread-launch.test.mjs:1:1)
✖ packages/cli/tests/thread-launch.test.mjs (40023.3188ms)
ℹ tests 3
ℹ suites 0
ℹ pass 0
ℹ fail 2
ℹ cancelled 1
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 40027.912798
✖ failing tests:
test at packages/cli/tests/thread-launch.test.mjs:167:1
✖ REQ-THREAD-2 REQ-LAUNCH-1 REQ-MSG-1 a thread message starts the thread's session, which stops when idle; its history survives the stop and a restart and no other thread sees it (20109.08171ms)
Error: timed out waiting
at until (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:48:9)
at runNextTicks (node:internal/process/task_queues:65:5)
at process.processTimers (node:internal/timers:615:9)
at async ask (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:153:17)
at async TestContext.<anonymous> (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:179:17)
at async Test.run (node:internal/test_runner/test:1409:7)
at async startSubtestAfterBootstrap (node:internal/test_runner/harness:387:3)
test at packages/cli/tests/thread-launch.test.mjs:216:1
✖ REQ-THREAD-2 REQ-LAUNCH-1 a running thread counts against its model's limit and an idle-stopped one doesn't (330.738356ms)
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected
+ 'launch-failed'
- 'capacity-full'
at TestContext.<anonymous> (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:227:10)
at runNextTicks (node:internal/process/task_queues:65:5)
at process.processTimers (node:internal/timers:615:9)
at async Test.run (node:internal/test_runner/test:1409:7)
at async Test.processPendingSubtests (node:internal/test_runner/test:974:7) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: 'launch-failed',
expected: 'capacity-full',
operator: 'strictEqual',
diff: 'simple'
}
test at packages/cli/tests/thread-launch.test.mjs:1:1
✖ packages/cli/tests/thread-launch.test.mjs (40023.3188ms)
'Promise resolution is still pending but the event loop has already resolved'
```
## sage-4204 — chief credential fixture correction (unrun)
Sage's control pair with a 29-byte TMPDIR: base 3e5041b43 passes 14/14;
ed985c044 passes all 14 existing tests but this row's new CLI fixture refuses
chief-library-gitea-required before launch. Added only chief.credentials.gitea
as a copy of the fixture coder's reference in the C2-05b edit callback, matching
cli/tests/launcher.test.mjs:35 and trackers-boot.test.mjs:17. This is a reference
to synthetic fixture credentials; no credential value was read or introduced.
All assertions, old tests and production code unchanged. No run performed.
The earlier 48-byte TMPDIR explains the launch-failed reds through the
107-byte tools.sock limit, as confirmed by Sage's control. Future named CLEAR
uses shell TMPDIR=$(mktemp -d /mnt/storage/scratch/tmp/gXXX), 29 bytes with
alphanumeric suffix, not Python mkdtemp or a canonical node_modules link.
Next authorized run order: whole thread-launch first, other eight files, whole
bus/harness suites, M-B1; all await next CLEAR after Fethry/Gyro slots.
## sage-4208 — first-file red, sequence stopped
Named candidate 6160b4981. Whole thread-launch.test.mjs invoked once.
Shell-created TMPDIR /mnt/storage/scratch/tmp/gCXV, 29 bytes; absolute runner
and timeout niceness 19 at start, Node observed 16 afterwards (host policy).
15 tests: all 14 existing tests pass, the one new C2-05b fails; no skips or
cancellations. Exit 1 in 18.740 s. Stopped sequence and returned slot to Sage
in della-0047 before any edit or rerun. No other file, suite or mutant ran.
The original r3 list has nine files beyond thread-launch (two row-owned plus
seven upstream), rather than eight; none started in this CLEAR.
New-test failure is call(...).find is not a function in existing ask helper
:153, called by new C2-05b :550. Root cause not yet established; no assertion,
fixture or production change made. Await Sage's own-new-test ruling.
Exact run receipt:
```text
{"number": 1, "name": "packages/cli/tests/thread-launch.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/cli/tests/thread-launch.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/gCXV", "start_nice": 19, "observed_timeout_nice": 19, "exit": 1, "seconds": 18.74}
```
Failure output:
```text
ℹ tests 15
ℹ suites 0
ℹ pass 14
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 18707.936662
✖ failing tests:
test at packages/cli/tests/thread-launch.test.mjs:540:1
✖ C2-05b a real launched bundle gets lookups for a library project without a library checkout (139.729167ms)
TypeError: call(...).find is not a function
at file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:153:71
at until (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:44:21)
at ask (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:153:23)
at TestContext.<anonymous> (file:///mnt/storage/scratch/della-row224-r2/packages/cli/tests/thread-launch.test.mjs:550:9)
at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
at async Test.run (node:internal/test_runner/test:1409:7)
at async Test.processPendingSubtests (node:internal/test_runner/test:974:7)
```
## sage-4210 — new-test async citation fixture fix
Only C2-05b now inlines send/wait, awaits thread.messages before .find, and
keeps the from_role assistant / to_role human assertions previously provided
by ask. Comment names broker.mjs:4854 async citation hydration. Existing ask,
old tests and production unchanged. This is Sage's own-new-test ruling under
mos-1590; Dewey re-reviews the delta. Whole-file rerun once authorized, then
the nine other files, whole bus/harness, M-B1 and clean-tree check on green.
Each gets a fresh shell gXXX TMPDIR and absolute nice 19; any red stops.
## sage-4210 — resumed sequence, harness new-test red
On 8f1b045f4: whole CLI 15/15 pass, no skips (19.164 s); whole bus lookup
11/11 pass, no skips (1.309 s). Own new harness file 1 pass, 3 fail, no skips
(0.129 s); stopped sequence, returned slot in della-0050 before edits/reruns.
Each invocation used fresh shell gXXX TMPDIR, 29 bytes. Runner was set to
absolute nice 19 before first spawn; host subsequently reniced runner too,
so timeout process observations were 19, -4, -4. Correct future driver to set
absolute 19 in each child before exec, as well as parent. Priority-only change
did not stop these runs. No claim that later timeout processes retained 19.
All three new harness failures are fixture validation: roles.bean.credentials
must reference gitea (got none). Shared role() helper in this row creates bean
without its required credential reference. No fixture/assertion/production edit
or rerun yet; awaiting Sage's ruling. Seven upstream files, whole bus/harness
suites and M-B1 not run. No full landing gate or in-review move.
Exact receipts:
```text
{"number": 1, "name": "packages/cli/tests/thread-launch.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/cli/tests/thread-launch.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/gBj5", "start_nice": 19, "observed_timeout_nice": 19, "exit": 0, "seconds": 19.164}
{"number": 2, "name": "packages/bus/tests/library-lookups.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/bus/tests/library-lookups.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/gD8X", "start_nice": 19, "observed_timeout_nice": -4, "exit": 0, "seconds": 1.309}
{"number": 3, "name": "packages/harness/tests/library-tools.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/harness/tests/library-tools.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/gt7o", "start_nice": 19, "observed_timeout_nice": -4, "exit": 1, "seconds": 0.129}
```
Harness failure output:
```text
✖ M3: per-role snapshots contain six names, or eight with the C2-08 web grant (6.427541ms)
✔ every noncreative role gets lookups when a library is configured, not otherwise (6.837154ms)
✖ lookup schemas require a universe even for get, include paging, and cannot select repo or commit (1.52147ms)
✖ bundle flag reaches both harness tool manifests and creative prompt without generic tools (2.019053ms)
ℹ tests 4
ℹ suites 0
ℹ pass 1
ℹ fail 3
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 99.111266
✖ failing tests:
test at packages/harness/tests/library-tools.test.mjs:19:1
✖ M3: per-role snapshots contain six names, or eight with the C2-08 web grant (6.427541ms)
Error [BusinessError]: /mnt/storage/scratch/tmp/gt7o/mosaic-harness-biz-CZEENl/businesses/acme.json roles.bean.credentials must reference exactly the services its role definition needs (gitea; got none)
at refuse (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/errors.mjs:12:9)
at checkCredentialMap (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/business.mjs:51:5)
at checkRoles (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/business.mjs:162:20)
at validateBusinessDocument (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/business.mjs:251:34)
at resolvedFor (file:///mnt/storage/scratch/della-row224-r2/packages/harness/tests/helpers.mjs:46:20)
at role (file:///mnt/storage/scratch/della-row224-r2/packages/harness/tests/library-tools.test.mjs:13:49)
at TestContext.<anonymous> (file:///mnt/storage/scratch/della-row224-r2/packages/harness/tests/library-tools.test.mjs:21:42)
at Test.runInAsyncScope (node:async_hooks:226:14)
at Test.run (node:internal/test_runner/test:1402:25)
at Test.start (node:internal/test_runner/test:1262:17) {
exitCode: 2
}
test at packages/harness/tests/library-tools.test.mjs:38:1
✖ lookup schemas require a universe even for get, include paging, and cannot select repo or commit (1.52147ms)
Error [BusinessError]: /mnt/storage/scratch/tmp/gt7o/mosaic-harness-biz-iAwXhO/businesses/acme.json roles.bean.credentials must reference exactly the services its role definition needs (gitea; got none)
at refuse (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/errors.mjs:12:9)
at checkCredentialMap (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/business.mjs:51:5)
at checkRoles (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/business.mjs:162:20)
at validateBusinessDocument (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/business.mjs:251:34)
at resolvedFor (file:///mnt/storage/scratch/della-row224-r2/packages/harness/tests/helpers.mjs:46:20)
at role (file:///mnt/storage/scratch/della-row224-r2/packages/harness/tests/library-tools.test.mjs:13:49)
at TestContext.<anonymous> (file:///mnt/storage/scratch/della-row224-r2/packages/harness/tests/library-tools.test.mjs:39:40)
at Test.runInAsyncScope (node:async_hooks:226:14)
at Test.run (node:internal/test_runner/test:1402:25)
at Test.processPendingSubtests (node:internal/test_runner/test:974:18) {
exitCode: 2
}
test at packages/harness/tests/library-tools.test.mjs:57:1
✖ bundle flag reaches both harness tool manifests and creative prompt without generic tools (2.019053ms)
Error [BusinessError]: /mnt/storage/scratch/tmp/gt7o/mosaic-harness-biz-c2N3Bv/businesses/acme.json roles.bean.credentials must reference exactly the services its role definition needs (gitea; got none)
at refuse (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/errors.mjs:12:9)
at checkCredentialMap (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/business.mjs:51:5)
at checkRoles (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/business.mjs:162:20)
at validateBusinessDocument (file:///mnt/storage/scratch/della-row224-r2/packages/business/src/business.mjs:251:34)
at resolvedFor (file:///mnt/storage/scratch/della-row224-r2/packages/harness/tests/helpers.mjs:46:20)
at role (file:///mnt/storage/scratch/della-row224-r2/packages/harness/tests/library-tools.test.mjs:13:49)
at TestContext.<anonymous> (file:///mnt/storage/scratch/della-row224-r2/packages/harness/tests/library-tools.test.mjs:60:65)
at Test.runInAsyncScope (node:async_hooks:226:14)
at Test.run (node:internal/test_runner/test:1402:25)
at Test.processPendingSubtests (node:internal/test_runner/test:974:18) {
exitCode: 2
}
```
## sage-4213 — creative fixture credential reference
Only the new library-tools.test.mjs role() helper now gives bean its required
gitea reference copied from coder, matching creative.test.mjs:27. No assertion,
shared helper, old test or production change. One whole harness-file retry
then seven upstream files, whole bus/harness, M-B1 and clean-tree check are
cleared; no CLI or lookup rerun needed (both passed, unchanged). Each child
sets absolute niceness 19 before exec, with fresh shell gXXX TMPDIR.
## sage-4213 — GREEN verification and M-B1 receipt
Verified candidate e4c0c8931; prior CLI/lookup passes on 8f1b045f4 remain
applicable under sage-4213 (only harness fixture and receipts changed).
Every run below used a fresh shell mktemp -d /mnt/storage/scratch/tmp/gXXX,
29 bytes. Each child set absolute niceness 19 immediately before exec; timeout
process observations were 19 for all ten runs and M-B1. Bus-suite Node later
observed at nice 16, consistent with host ananicy policy; no stop for nice.
No unexpected red, cancellation or skip in this cleared sequence.
Whole-file results:
| File or suite | Pass | Fail | Skip | Seconds |
| --- | ---: | ---: | ---: | ---: |
| packages/harness/tests/library-tools.test.mjs | 4 | 0 | 0 | 0.213 |
| packages/bus/tests/room-check.test.mjs | 2 | 0 | 0 | 0.21 |
| packages/plugins/tests/room-channel.test.mjs | 1 | 0 | 0 | 0.102 |
| packages/supervisor/tests/pressure.test.mjs | 15 | 0 | 0 | 0.101 |
| packages/supervisor/tests/purity.test.mjs | 1 | 0 | 0 | 0.102 |
| packages/supervisor/tests/rules.test.mjs | 29 | 0 | 0 | 0.102 |
| packages/webui/tests/room-browser.test.mjs | 9 | 0 | 0 | 6.801 |
| packages/webui/tests/room-http.test.mjs | 7 | 0 | 0 | 0.183 |
| bus-whole | 1011 | 0 | 0 | 30.46 |
| harness-whole | 176 | 0 | 0 | 13.346 |
Earlier cleared whole CLI 15/15 (19.164 s), bus lookup 11/11 (1.309 s),
zero skips. Whole bus includes lookup again; whole harness includes fixture
file again. No full shell/landing gate run: every scripts/test-*.sh belongs
to Sage's landing gate, not this CLEAR.
M-B1: temporarily replaced only lastError.at freshness term with -Infinity.
Whole lookup file: 10 pass, 1 expected fail, no skips; killed at :178 attempted
assertion (actual 2, expected 1, recent lastError prevents another sync).
Restored exact original runtime bytes in finally. Afterwards git diff --stat
and git status --porcelain both empty against committed e4c0c8931, before
adding this receipt. Compared with named 6160b4981 the only later package
changes are Sage-authorized async CLI fixture and harness credential fixture;
no mutant or production change remains. No extra post-restore rerun.
Exact commands, TMPDIRs, exit codes and elapsed receipts:
```text
{"number": 1, "name": "packages/harness/tests/library-tools.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/harness/tests/library-tools.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/gct3", "start_nice": 19, "observed_timeout_nice": 19, "exit": 0, "seconds": 0.213}
{"number": 2, "name": "packages/bus/tests/room-check.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/bus/tests/room-check.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/g0ki", "start_nice": 19, "observed_timeout_nice": 19, "exit": 0, "seconds": 0.21}
{"number": 3, "name": "packages/plugins/tests/room-channel.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/plugins/tests/room-channel.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/gyTq", "start_nice": 19, "observed_timeout_nice": 19, "exit": 0, "seconds": 0.102}
{"number": 4, "name": "packages/supervisor/tests/pressure.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/supervisor/tests/pressure.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/gK1n", "start_nice": 19, "observed_timeout_nice": 19, "exit": 0, "seconds": 0.101}
{"number": 5, "name": "packages/supervisor/tests/purity.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/supervisor/tests/purity.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/gFne", "start_nice": 19, "observed_timeout_nice": 19, "exit": 0, "seconds": 0.102}
{"number": 6, "name": "packages/supervisor/tests/rules.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/supervisor/tests/rules.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/gs9Y", "start_nice": 19, "observed_timeout_nice": 19, "exit": 0, "seconds": 0.102}
{"number": 7, "name": "packages/webui/tests/room-browser.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/webui/tests/room-browser.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/gP9P", "start_nice": 19, "observed_timeout_nice": 19, "exit": 0, "seconds": 6.801}
{"number": 8, "name": "packages/webui/tests/room-http.test.mjs", "command": ["timeout", "-k", "10", "900", "node", "--test", "packages/webui/tests/room-http.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/gP9G", "start_nice": 19, "observed_timeout_nice": 19, "exit": 0, "seconds": 0.183}
{"number": 9, "name": "bus-whole", "command": ["timeout", "-k", "10", "1800", "node", "--test", "packages/bus/tests/access-revoke.test.mjs", "packages/bus/tests/account-migration.test.mjs", "packages/bus/tests/address.test.mjs", "packages/bus/tests/agents.test.mjs", "packages/bus/tests/approval-binding.test.mjs", "packages/bus/tests/broker.test.mjs", "packages/bus/tests/business.test.mjs", "packages/bus/tests/channel-proof.test.mjs", "packages/bus/tests/chief-of-staff.test.mjs", "packages/bus/tests/citation-availability.test.mjs", "packages/bus/tests/citation-guard.test.mjs", "packages/bus/tests/citation-runtime.test.mjs", "packages/bus/tests/comment-cursors.test.mjs", "packages/bus/tests/comment-migration.test.mjs", "packages/bus/tests/console-decide.test.mjs", "packages/bus/tests/creative.test.mjs", "packages/bus/tests/credential-claim.test.mjs", "packages/bus/tests/credential-status.test.mjs", "packages/bus/tests/credential-store.test.mjs", "packages/bus/tests/credential-verbs.test.mjs", "packages/bus/tests/credentials.test.mjs", "packages/bus/tests/csuite-messaging.test.mjs", "packages/bus/tests/daily-quota-runtime.test.mjs", "packages/bus/tests/decision-close.test.mjs", "packages/bus/tests/decision-route.test.mjs", "packages/bus/tests/decisions.test.mjs", "packages/bus/tests/end-launch.test.mjs", "packages/bus/tests/fire-revokes-grants.test.mjs", "packages/bus/tests/gitea-grant-docker.test.mjs", "packages/bus/tests/gitea-grant.test.mjs", "packages/bus/tests/gitea-migration.test.mjs", "packages/bus/tests/grant-enroll.test.mjs", "packages/bus/tests/grant-executor.test.mjs", "packages/bus/tests/grant-host.test.mjs", "packages/bus/tests/grant-migration.test.mjs", "packages/bus/tests/grant-scope-runtime.test.mjs", "packages/bus/tests/grant-scope.test.mjs", "packages/bus/tests/hard-waits.test.mjs", "packages/bus/tests/host-run.test.mjs", "packages/bus/tests/host-tools.test.mjs", "packages/bus/tests/human.test.mjs", "packages/bus/tests/instances-verb.test.mjs", "packages/bus/tests/interpreter.test.mjs", "packages/bus/tests/land.test.mjs", "packages/bus/tests/launch-notice.test.mjs", "packages/bus/tests/launcher-released.test.mjs", "packages/bus/tests/library-credential.test.mjs", "packages/bus/tests/library-lookups.test.mjs", "packages/bus/tests/library-sync.test.mjs", "packages/bus/tests/library.test.mjs", "packages/bus/tests/lifecycle-revoke.test.mjs", "packages/bus/tests/lifecycle-verbs.test.mjs", "packages/bus/tests/limit-checks.test.mjs", "packages/bus/tests/limit-stop.test.mjs", "packages/bus/tests/limits-runtime.test.mjs", "packages/bus/tests/limits.test.mjs", "packages/bus/tests/message-citation.test.mjs", "packages/bus/tests/message-filter.test.mjs", "packages/bus/tests/messages.test.mjs", "packages/bus/tests/messaging.test.mjs", "packages/bus/tests/name-migration.test.mjs", "packages/bus/tests/names.test.mjs", "packages/bus/tests/peer-transport.test.mjs", "packages/bus/tests/peercred.test.mjs", "packages/bus/tests/plugins-verb.test.mjs", "packages/bus/tests/principal.test.mjs", "packages/bus/tests/process.test.mjs", "packages/bus/tests/processing-runtime.test.mjs", "packages/bus/tests/processing-timeout.test.mjs", "packages/bus/tests/processing-web.test.mjs", "packages/bus/tests/processing.test.mjs", "packages/bus/tests/project-await.test.mjs", "packages/bus/tests/project-files.test.mjs", "packages/bus/tests/project-runtime.test.mjs", "packages/bus/tests/publish.test.mjs", "packages/bus/tests/reload.test.mjs", "packages/bus/tests/replies.test.mjs", "packages/bus/tests/requirements.test.mjs", "packages/bus/tests/review.test.mjs", "packages/bus/tests/revoke-window.test.mjs", "packages/bus/tests/room-check.test.mjs", "packages/bus/tests/rooms.test.mjs", "packages/bus/tests/routes-verb.test.mjs", "packages/bus/tests/schema.test.mjs", "packages/bus/tests/session-clear.test.mjs", "packages/bus/tests/session-compactions.test.mjs", "packages/bus/tests/sessions.test.mjs", "packages/bus/tests/single-use.test.mjs", "packages/bus/tests/spec-migration.test.mjs", "packages/bus/tests/spec-review.test.mjs", "packages/bus/tests/spec-socket.test.mjs", "packages/bus/tests/spec.test.mjs", "packages/bus/tests/staff-request.test.mjs", "packages/bus/tests/state.test.mjs", "packages/bus/tests/store.test.mjs", "packages/bus/tests/task-page.test.mjs", "packages/bus/tests/task-read.test.mjs", "packages/bus/tests/tasks.test.mjs", "packages/bus/tests/tester-threads.test.mjs", "packages/bus/tests/tester-writes.test.mjs", "packages/bus/tests/tester.test.mjs", "packages/bus/tests/thread-claims.test.mjs", "packages/bus/tests/thread-migration.test.mjs", "packages/bus/tests/thread-shares.test.mjs", "packages/bus/tests/threads.test.mjs", "packages/bus/tests/transport.test.mjs", "packages/bus/tests/usage-broker.test.mjs", "packages/bus/tests/usage-migration.test.mjs", "packages/bus/tests/usage-runtime.test.mjs", "packages/bus/tests/usage.test.mjs", "packages/bus/tests/user-principal.test.mjs", "packages/bus/tests/viewer.test.mjs", "paLine truncated
{"number": 10, "name": "harness-whole", "command": ["timeout", "-k", "10", "1800", "node", "--test", "packages/harness/tests/bundle.test.mjs", "packages/harness/tests/citation-runner.test.mjs", "packages/harness/tests/claude-gate.test.mjs", "packages/harness/tests/claude-session.test.mjs", "packages/harness/tests/creative.test.mjs", "packages/harness/tests/gate.test.mjs", "packages/harness/tests/git-tools.test.mjs", "packages/harness/tests/library-tools.test.mjs", "packages/harness/tests/mcp-server.test.mjs", "packages/harness/tests/pi-session.test.mjs", "packages/harness/tests/raise-domain.test.mjs", "packages/harness/tests/runner.test.mjs", "packages/harness/tests/skills.test.mjs", "packages/harness/tests/spec-tools.test.mjs", "packages/harness/tests/tools.test.mjs"], "TMPDIR": "/mnt/storage/scratch/tmp/g8Yn", "start_nice": 19, "observed_timeout_nice": 19, "exit": 0, "seconds": 13.346}
{
"mutant": "M-B1",
"change": "drop lastError.at term from recentAttempt",
"command": [
"timeout",
"-k",
"10",
"900",
"node",
"--test",
"packages/bus/tests/library-lookups.test.mjs"
],
"TMPDIR": "/mnt/storage/scratch/tmp/g8my",
"start_nice": 19,
"observed_timeout_nice": 19,
"exit": 1,
"seconds": 1.359
}
$ git diff --stat
(empty)
$ git status --porcelain
(empty)
HEAD=e4c0c8931 (authorized fixture fix after 6160b4981)
```
M-B1 output:
```text
✔ all five agent verbs read the configured served commit through real queries (130.428728ms)
✔ each verb refuses missing universe and unknown keys before host work (78.900515ms)
✔ no library, missing host adapter and undeclared project refuse library-not-configured (149.899073ms)
✔ reads need the current holder on each call; human/tester/reader are not agents (90.636537ms)
✔ releasing a claim during an async lookup fences the response (60.938885ms)
✔ a business library removed during an async read cannot leak its result (51.783488ms)
✖ freshness boundary, unknown last fetch and failed fetch still use immutable served state (123.453393ms)
✔ an unreadable sync state fails closed; a busy sync keeps serving the existing pin (89.763849ms)
✔ a stale lookup sees the commit published by its sync, not the prior served commit (108.213675ms)
✔ real socket awaits lookup results and rechecks rejected query arguments (152.413684ms)
✔ runtime actually injects the lookup adapter, without exposing human library.file (157.701866ms)
ℹ tests 11
ℹ suites 0
ℹ pass 10
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 1327.433364
✖ failing tests:
test at packages/bus/tests/library-lookups.test.mjs:153:1
✖ freshness boundary, unknown last fetch and failed fetch still use immutable served state (123.453393ms)
AssertionError [ERR_ASSERTION]: recent lastError prevents another sync even though lastFetch is null
2 !== 1
at TestContext.<anonymous> (file:///mnt/storage/scratch/della-row224-r2/packages/bus/tests/library-lookups.test.mjs:178:12)
at async Test.run (node:internal/test_runner/test:1409:7)
at async Test.processPendingSubtests (node:internal/test_runner/test:974:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: 2,
expected: 1,
operator: 'strictEqual',
diff: 'simple'
}
```
+166
View File
@@ -0,0 +1,166 @@
# Row 257: Q198 S1 pressure parser and pure rules evaluator
Issue #1762. Brief: `docs/plans/2026-10-21_post-v1-row-briefs.md`, "Brief: Q198 S1 pressure parser and
pure rules evaluator" and "Gate: Q198 S1" (cca206cdc). Design: `docs/plans/2026-10-21_q198-system-manager.md`
r6.1, section 8 row S1 and section 4 (D4). Owner Morgana, reviewer Filbert, gate owner Sage (sage-2196).
Post-v1: MVP rows and re-pin windows go ahead of it on the slot. Base: refactor cca206cdc.
## Recorded choices (sage-2196 accepted each)
1. **Placement.** A new package, `packages/supervisor` (`@mosaic/supervisor`), shaped like
packages/runs. Nothing on refactor parsed loadavg, PSI, meminfo or cgroup limits (`git grep`; the
only hit was an awk line in scripts/case2-precheck.sh). S2 adds the sampler process to the same
package. Packages aren't registered centrally, so no other file changes.
2. **Provider scope** (429, quota, Retry-After, the half-open probe) is out. The brief puts it in S3
and S5.
3. **PSI** is not required on a host that has never shown it, so a kernel built without PSI skips the
PSI rows instead of holding forever. Once a sample has carried PSI, it is required: a later missing
or stale PSI signal holds with `health-unknown`. That is the case sage-2196 asked for.
- Tradeoff: I latch on the first sample that carries PSI, not only on the very first sample. So PSI
that appears late and then disappears also holds. This is stricter than "present at the first
sample", and it never skips a signal the host has shown.
4. **Memory under a cgroup.** Available memory = min(MemAvailable, limit - memory.current) at each
limited level. memory.current counts page cache, so this errs toward holding.
- Follow-up, not in this row: subtract reclaimable cache using memory.stat `inactive_file`, if S2's
live samples show holds that the host's MemAvailable doesn't support.
Rulings on two readings from the build (sage-2202, re morgana-0057):
5. **PSI latch.** PSI becomes required on the first sample that carries it. This is stricter than
"present at the first sample" and fails closed: a PSI file that disappears after it was seen is a
missing signal, so health-unknown holds.
6. **PSI count across resources.** Design line 152 says "CPU some ≥20, memory full ≥2, or I/O full
≥10, for 3 samples". The window applies to the whole disjunction, so any PSI condition on 3
consecutive samples holds. If Filbert reads it per resource at review, the review finding decides,
and the fix is a single counter.
Other readings of the D4 table, made in the build:
- **Memory floors.** "Whichever demands more free memory" means the larger floor applies. Hold below
max(15% x limit, 1 GiB); release above max(25% x limit, 2 GiB).
- **Edges as written.** Hold at ≥ for load and PSI, < for memory. Release at ≤ 0.55 for load, < for
PSI, > for memory. Critical below 5% of the limit and below 1 GiB of disk.
- **Sample counts.** "For 3 samples" means 3 consecutive samples. The count resets after a gap of more
than 6 s (the stale limit) between samples. The PSI row counts as one row (ruling 6), and the
reasons name the resources over on the latest sample.
- **Memory holds on one sample.** The table's memory row has no sample count.
- **Release** needs every applicable release condition on every sample for 30 s, by the samples' own
times (`>= 30000` ms after the first qualifying sample). A dead-band sample or any hold restarts the
window, and so does a gap of more than 6 s.
- **Start held.** `initialState()` is held, because nothing has been seen yet. The first healthy
sample starts the 30 s window.
- **Missing or stale.** Required signals are load, memory, oom and disk, plus PSI once seen. A signal
holds at once with `health-unknown` if it is:
- missing;
- non-numeric, negative or zero where a divisor;
- more than 6 s older than its sample (6 s exactly is fresh);
- stamped later than its sample.
Disk and OOM are required because the critical row can't be evaluated without them.
- **Host-only OOM.** `/proc/vmstat` `oom_kill` gives the OOM counter when there's no cgroup
memory.events. The parser returns null for a kernel without it, and the sampler then has no OOM
signal (health-unknown).
- **Critical** latches. It ends only when the caller passes `reconciled` with no critical condition in
the sample, then goes through held and the normal 30 s release. The evaluator never sets
`reconciled` and has no kill-switch path. Critical reasons accumulate while latched.
- **OOM counter.** The first OOM sample is the baseline. A falling counter (a recreated cgroup) resets
the baseline with a note, not an event.
- **Clock going backwards.** A sample earlier than the last one holds with `health-unknown` (unknown
`clock`, note `clock-backwards`).
- **Swap activity** is parsed (`pswpin`, `pswpout`) and has no rule. D4 lists it as a signal with no
threshold.
- **Settings.** `resolveRules` refuses:
- unknown keys and non-numbers;
- a release at or past its hold (load, each PSI resource);
- memory release floors at or below the hold floors;
- fractions outside (0, 1);
- PSI values outside 0 to 100;
- sample counts that aren't integers ≥ 1;
- a negative release window;
- a stale limit at or below the sample interval;
- a critical memory fraction at or above the hold fraction;
- a non-positive disk floor.
- **Large byte values.** Parsed with `Number`. A memory.max above 2^53 loses precision, but it only
takes part in a min against MemTotal.
## Round 2 (Filbert r1 APPROVE c30188 notes, sage-2212)
- N1: `merge` checked unknown keys with `key in base`, so inherited names (`toString`, `constructor`,
`__proto__`) passed the check and were dropped instead of refused. It now uses
`Object.hasOwn(base, key)`, and the refusal test has those three cases.
- N2: the critical-memory edge was tested only on a 32 GiB limit, where 5% is not an integer, so a
`<` to `<=` mutant survived. The test adds a 20 GiB limit, where 5% is exactly 1 GiB, and M8 is that
mutant. I checked the other memory edges for the same gap:
- The hold floor is non-integer on 32 GiB but is tested at exactly 1 GiB on a 4 GiB limit.
- The release floors are 8 GiB on 32 GiB and 2 GiB on 4 GiB, both integers.
- The disk edge is 1 GiB.
None has the gap.
## Files
- `packages/supervisor/package.json`, `README.md`
- `src/errors.mjs`: `SupervisorError` with `kind` parse, rules or sample.
- `src/pressure.mjs`: the nine parsers, `effectiveCpus` and `effectiveMemory`.
- `src/rules.mjs`: `DEFAULT_RULES`, `resolveRules`, `initialState`, `evaluate`.
- `src/index.mjs`
- `tests/pressure.test.mjs`: parsers on kernel-shaped fixture text, with malformed, truncated and
repeated-key cases. Effective CPU cases cover the host-only fallback, cpuset, fractional and nested
`cpu.max`, and `max`. Effective memory cases cover the host-only fallback, `max`, nested levels,
usage over the limit, a limit above MemTotal and a missing memory.current.
- `tests/rules.test.mjs`: the D4 table on a fake clock.
- Each hold and release threshold on both sides of its edge.
- Three-sample counting and the gap reset.
- The 30 s release at 29 999 and 30 000 ms, and the dead-band restart.
- Missing, stale, future and malformed signals.
- PSI never present, and PSI lost after it was present.
- Critical OOM, memory and disk at their edges, the reconciliation latch and the counter reset.
- A backwards clock, input state left unchanged, and settings refusals and overrides.
- `tests/purity.test.mjs`: a static check that src imports only its own files and uses no
fs, child_process, net, os, process, dynamic import, Date or timers.
No test reads /proc, /sys or a cgroup, starts a build, opens a network socket or generates load. No new
dependency.
## Mutants (each must fail its named test; the brief requires M1 and M2)
| Mutant | Edit | Named test |
|---|---|---|
| M1 hold edge | `ratio >= rules.load.hold` becomes `>` | load holds at 0.80 for 3 samples and not at 0.7999 |
| M2 early release | `now - next.releaseSince >= rules.releaseMs` becomes `>= 0` | the first state is held, and healthy samples open it after exactly 30 s |
| M3 memory floor | the hold floor `Math.max` becomes `Math.min` | memory holds below the larger floor, on one sample |
| M4 sample count | `next.loadCount >= rules.load.samples` becomes `>= 1` | load holds at 0.80 for 3 samples and not at 0.7999 |
| M5 stale | the `now - signal.at > rules.staleMs` check is removed | a signal is stale at more than 6 s old and fresh at exactly 6 s |
| M6 effective CPU | the quota `Math.min` becomes `Math.max` | effective CPUs take the smallest of host, cpuset and every quota level |
| M7 critical latch | the `!reconciled` critical branch is removed | critical stays until reconciliation, then still waits the 30 s release |
| M8 critical memory edge (r2) | `available < rules.critical.memoryFraction * limit` becomes `<=` | memory under 5% and disk under 1 GiB are critical, at their edges |
## Gate (Sage's slot, on CLEAR only)
- The three test files, each whole, then `node --test tests/` in packages/supervisor as the package
suite.
- Then M1 to M8, each on its named test, restored after each.
- umask 022, TMPDIR from `mktemp -d /mnt/storage/scratch/tmp/gXXXXXX`.
- The kit is /mnt/storage/scratch/morgana-r257 (slot257.sh, mutate.py). It refuses unless HEAD is the
candidate and the tree is clean, and stops after two unexpected failures (mos-0823).
- Static checks before the slot: `node --check` on every file, run from `/`. No test has run yet.
## Slot result (sage-4159 CLEAR, 2026-10-24 17:00:27Z to 17:00:30Z, candidate 1261fa2d8)
Run as `busy.sh && timeout -k 10 1500 slot257.sh` in its own session (setsid). The busy check read
CLEAR. The worktree was checked out at 1261fa2d8 with a clean tree. Filbert's approved code is
unchanged. The summary is in `gate-run.log` and the per-run node output in `mutants-run.log`.
| Run | tests | pass | fail | skip | cancelled | todo |
|---|---|---|---|---|---|---|
| tests/pressure.test.mjs whole | 15 | 15 | 0 | 0 | 0 | 0 |
| tests/rules.test.mjs whole | 29 | 29 | 0 | 0 | 0 | 0 |
| tests/purity.test.mjs whole | 1 | 1 | 0 | 0 | 0 | 0 |
| package suite `node --test tests/` | 45 | 45 | 0 | 0 | 0 | 0 |
Mutants: 8 of 8 killed. Each one ran only its named test (1 test, rc=1), and each failure was an
AssertionError in that test, not a crash: M1 hold edge, M2 early release, M3 memory floor, M4 sample
count, M5 stale, M6 effective CPU, M7 critical latch, M8 critical memory edge.
Restore check: `git checkout -- packages/supervisor` ran after each mutant, and the script found the
tree clean every time. At the end HEAD was 1261fa2d8 and `git status --porcelain` was empty. After the
run no node, python or bash process had its cwd under the kit or the worktree, and no g257 TMPDIR was
left behind. The run reached no scripts/test-*.sh.
@@ -0,0 +1,34 @@
CLEAR
== files 2026-10-24T17:00:27Z
file-pressure rc=0 1s ℹ pass 15 ℹ fail 0
file-rules rc=0 0s ℹ pass 29 ℹ fail 0
file-purity rc=0 0s ℹ pass 1 ℹ fail 0
== package suite 2026-10-24T17:00:28Z
suite-supervisor rc=0 0s ℹ pass 45 ℹ fail 0
== mx 2026-10-24T17:00:28Z (each must fail its named test)
M1 applied to rules.mjs
mx-M1 rc=1 0s ℹ pass 0 ℹ fail 1
mx-M1 KILLED
M2 applied to rules.mjs
mx-M2 rc=1 0s ℹ pass 0 ℹ fail 1
mx-M2 KILLED
M3 applied to rules.mjs
mx-M3 rc=1 0s ℹ pass 0 ℹ fail 1
mx-M3 KILLED
M4 applied to rules.mjs
mx-M4 rc=1 0s ℹ pass 0 ℹ fail 1
mx-M4 KILLED
M5 applied to rules.mjs
mx-M5 rc=1 0s ℹ pass 0 ℹ fail 1
mx-M5 KILLED
M6 applied to pressure.mjs
mx-M6 rc=1 0s ℹ pass 0 ℹ fail 1
mx-M6 KILLED
M7 applied to rules.mjs
mx-M7 rc=1 0s ℹ pass 0 ℹ fail 1
mx-M7 KILLED
M8 applied to rules.mjs
mx-M8 rc=1 0s ℹ pass 0 ℹ fail 1
mx-M8 KILLED
restored at 1261fa2d8
SLOT257_DONE 2026-10-24T17:00:30Z
@@ -0,0 +1,391 @@
===== file-pressure.log
✔ loadavg parses the three loads (1.192277ms)
✔ a malformed loadavg throws (0.412303ms)
✔ meminfo returns bytes and keeps unitless counts (0.258214ms)
✔ meminfo without MemAvailable, with a repeated key or a bad line throws (0.206592ms)
✔ PSI parses some and full, and cpu may lack full (0.374747ms)
✔ PSI without a full line for memory or io, or with a bad line, throws (0.195229ms)
✔ CPU lists count distinct CPUs (0.237299ms)
✔ cpu.max gives the quota in CPUs, or null for max (0.23685ms)
✔ memory.max, memory.current and memory.events (0.351243ms)
✔ vmstat gives swap counters and the host OOM counter (0.234271ms)
✔ effective CPUs fall back to the host count with no cgroup input (0.119728ms)
✔ effective CPUs take the smallest of host, cpuset and every quota level (0.117052ms)
✔ effective memory falls back to the host with no cgroup levels, or only max (0.609639ms)
✔ effective memory uses the tightest limited level (0.187466ms)
✔ a limited level without memory.current throws (0.096904ms)
ℹ tests 15
ℹ suites 0
ℹ pass 15
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 171.66419
===== file-rules.log
✔ the first state is held, and healthy samples open it after exactly 30 s (2.048214ms)
✔ load holds at 0.80 for 3 samples and not at 0.7999 (1.121376ms)
✔ load divides by effective CPUs, fractional ones included (0.360874ms)
✔ a gap of more than 6 s resets the consecutive count (0.343221ms)
✔ load releases at 0.55 and not at 0.5501 (0.435882ms)
✔ a dead-band sample restarts the 30 s release window (0.376884ms)
✔ memory holds below the larger floor, on one sample (0.259125ms)
✔ on a small limit the 1 GiB floor is the larger one (0.346965ms)
✔ memory releases only above the larger release floor (0.906516ms)
✔ each PSI hold threshold holds at its value for 3 samples and not just below (0.661209ms)
✔ the PSI count runs across resources (0.15624ms)
✔ each PSI release threshold blocks at its value and releases just below (0.613751ms)
✔ a missing required signal holds at once with health-unknown (0.388768ms)
✔ a signal is stale at more than 6 s old and fresh at exactly 6 s (0.446828ms)
✔ malformed signal values count as missing (0.787389ms)
✔ after health-unknown, recovery waits the full 30 s (0.34472ms)
✔ a host that never had PSI skips the PSI rows (0.201995ms)
✔ PSI that disappears after it was present holds with health-unknown (0.634442ms)
✔ a rise in the OOM kill counter is critical; the first sample is the baseline (0.230518ms)
✔ a falling OOM counter resets the baseline with a note, not an event (0.208182ms)
✔ memory under 5% and disk under 1 GiB are critical, at their edges (0.800954ms)
✔ critical stays until reconciliation, then still waits the 30 s release (0.585483ms)
✔ critical reasons accumulate while critical (0.174367ms)
✔ a clock that goes backwards holds with health-unknown (0.165886ms)
✔ a sample without a numeric time throws (0.515819ms)
✔ evaluate doesn't change the state it was given (0.327166ms)
✔ resolveRules with no overrides is the D4 table, frozen (0.502534ms)
✔ overrides apply in the evaluator (0.291524ms)
✔ contradictory or unknown settings refuse (1.168994ms)
ℹ tests 29
ℹ suites 0
ℹ pass 29
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 86.686704
===== file-purity.log
✔ src reads no file, process, network or clock (1.611513ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 63.38547
===== suite-supervisor.log
✔ loadavg parses the three loads (1.322312ms)
✔ a malformed loadavg throws (0.491714ms)
✔ meminfo returns bytes and keeps unitless counts (0.306568ms)
✔ meminfo without MemAvailable, with a repeated key or a bad line throws (0.282108ms)
✔ PSI parses some and full, and cpu may lack full (0.382364ms)
✔ PSI without a full line for memory or io, or with a bad line, throws (0.229427ms)
✔ CPU lists count distinct CPUs (0.310693ms)
✔ cpu.max gives the quota in CPUs, or null for max (0.258658ms)
✔ memory.max, memory.current and memory.events (0.36499ms)
✔ vmstat gives swap counters and the host OOM counter (0.27834ms)
✔ effective CPUs fall back to the host count with no cgroup input (0.148614ms)
✔ effective CPUs take the smallest of host, cpuset and every quota level (0.118641ms)
✔ effective memory falls back to the host with no cgroup levels, or only max (0.189321ms)
✔ effective memory uses the tightest limited level (0.248126ms)
✔ a limited level without memory.current throws (0.174502ms)
✔ src reads no file, process, network or clock (1.698537ms)
✔ the first state is held, and healthy samples open it after exactly 30 s (2.722678ms)
✔ load holds at 0.80 for 3 samples and not at 0.7999 (1.31022ms)
✔ load divides by effective CPUs, fractional ones included (0.457608ms)
✔ a gap of more than 6 s resets the consecutive count (0.487291ms)
✔ load releases at 0.55 and not at 0.5501 (0.471881ms)
✔ a dead-band sample restarts the 30 s release window (0.407363ms)
✔ memory holds below the larger floor, on one sample (0.311888ms)
✔ on a small limit the 1 GiB floor is the larger one (0.337859ms)
✔ memory releases only above the larger release floor (0.852137ms)
✔ each PSI hold threshold holds at its value for 3 samples and not just below (0.670722ms)
✔ the PSI count runs across resources (0.168238ms)
✔ each PSI release threshold blocks at its value and releases just below (0.706245ms)
✔ a missing required signal holds at once with health-unknown (0.45638ms)
✔ a signal is stale at more than 6 s old and fresh at exactly 6 s (0.501209ms)
✔ malformed signal values count as missing (0.703616ms)
✔ after health-unknown, recovery waits the full 30 s (0.252741ms)
✔ a host that never had PSI skips the PSI rows (0.136341ms)
✔ PSI that disappears after it was present holds with health-unknown (0.302628ms)
✔ a rise in the OOM kill counter is critical; the first sample is the baseline (0.181417ms)
✔ a falling OOM counter resets the baseline with a note, not an event (0.166744ms)
✔ memory under 5% and disk under 1 GiB are critical, at their edges (0.59429ms)
✔ critical stays until reconciliation, then still waits the 30 s release (0.346404ms)
✔ critical reasons accumulate while critical (0.111712ms)
✔ a clock that goes backwards holds with health-unknown (0.098686ms)
✔ a sample without a numeric time throws (0.425754ms)
✔ evaluate doesn't change the state it was given (0.239173ms)
✔ resolveRules with no overrides is the D4 table, frozen (0.368226ms)
✔ overrides apply in the evaluator (0.251687ms)
✔ contradictory or unknown settings refuse (0.917827ms)
ℹ tests 45
ℹ suites 0
ℹ pass 45
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 92.173971
===== mx-M1.log
✖ load holds at 0.80 for 3 samples and not at 0.7999 (2.627622ms)
ℹ tests 1
ℹ suites 0
ℹ pass 0
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 53.826624
✖ failing tests:
test at tests/rules.test.mjs:75:1
✖ load holds at 0.80 for 3 samples and not at 0.7999 (2.627622ms)
AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
+ actual - expected
[
'open',
'open',
+ 'open'
- 'held'
]
at TestContext.<anonymous> (file:///home/jwoltje/.t3/worktrees/mosaic-stack/morgana-row257/packages/supervisor/tests/rules.test.mjs:77:10)
at Test.runInAsyncScope (node:async_hooks:226:14)
at Test.run (node:internal/test_runner/test:1402:25)
at Test.start (node:internal/test_runner/test:1262:17)
at startSubtestAfterBootstrap (node:internal/test_runner/harness:387:17) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: [ 'open', 'open', 'open' ],
expected: [ 'open', 'open', 'held' ],
operator: 'deepStrictEqual',
diff: 'simple'
}
===== mx-M2.log
✖ the first state is held, and healthy samples open it after exactly 30 s (1.670671ms)
ℹ tests 1
ℹ suites 0
ℹ pass 0
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 48.463372
✖ failing tests:
test at tests/rules.test.mjs:62:1
✖ the first state is held, and healthy samples open it after exactly 30 s (1.670671ms)
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
'open' !== 'held'
at TestContext.<anonymous> (file:///home/jwoltje/.t3/worktrees/mosaic-stack/morgana-row257/packages/supervisor/tests/rules.test.mjs:65:10)
at Test.runInAsyncScope (node:async_hooks:226:14)
at Test.run (node:internal/test_runner/test:1402:25)
at Test.start (node:internal/test_runner/test:1262:17)
at startSubtestAfterBootstrap (node:internal/test_runner/harness:387:17) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: 'open',
expected: 'held',
operator: 'strictEqual',
diff: 'simple'
}
===== mx-M3.log
✖ memory holds below the larger floor, on one sample (2.628385ms)
ℹ tests 1
ℹ suites 0
ℹ pass 0
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 54.087261
✖ failing tests:
test at tests/rules.test.mjs:119:1
✖ memory holds below the larger floor, on one sample (2.628385ms)
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
'open' !== 'held'
at TestContext.<anonymous> (file:///home/jwoltje/.t3/worktrees/mosaic-stack/morgana-row257/packages/supervisor/tests/rules.test.mjs:125:10)
at Test.runInAsyncScope (node:async_hooks:226:14)
at Test.run (node:internal/test_runner/test:1402:25)
at Test.start (node:internal/test_runner/test:1262:17)
at startSubtestAfterBootstrap (node:internal/test_runner/harness:387:17) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: 'open',
expected: 'held',
operator: 'strictEqual',
diff: 'simple'
}
===== mx-M4.log
✖ load holds at 0.80 for 3 samples and not at 0.7999 (2.72525ms)
ℹ tests 1
ℹ suites 0
ℹ pass 0
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 54.494912
✖ failing tests:
test at tests/rules.test.mjs:75:1
✖ load holds at 0.80 for 3 samples and not at 0.7999 (2.72525ms)
AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
+ actual - expected
[
- 'open',
- 'open',
'held',
+ 'held',
+ 'held'
]
at TestContext.<anonymous> (file:///home/jwoltje/.t3/worktrees/mosaic-stack/morgana-row257/packages/supervisor/tests/rules.test.mjs:77:10)
at Test.runInAsyncScope (node:async_hooks:226:14)
at Test.run (node:internal/test_runner/test:1402:25)
at Test.start (node:internal/test_runner/test:1262:17)
at startSubtestAfterBootstrap (node:internal/test_runner/harness:387:17) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: [ 'held', 'held', 'held' ],
expected: [ 'open', 'open', 'held' ],
operator: 'deepStrictEqual',
diff: 'simple'
}
===== mx-M5.log
✖ a signal is stale at more than 6 s old and fresh at exactly 6 s (2.946775ms)
ℹ tests 1
ℹ suites 0
ℹ pass 0
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 58.672563
✖ failing tests:
test at tests/rules.test.mjs:187:1
✖ a signal is stale at more than 6 s old and fresh at exactly 6 s (2.946775ms)
AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
+ actual - expected
+ []
- [
- 'load'
- ]
at TestContext.<anonymous> (file:///home/jwoltje/.t3/worktrees/mosaic-stack/morgana-row257/packages/supervisor/tests/rules.test.mjs:191:10)
at Test.runInAsyncScope (node:async_hooks:226:14)
at Test.run (node:internal/test_runner/test:1402:25)
at Test.start (node:internal/test_runner/test:1262:17)
at startSubtestAfterBootstrap (node:internal/test_runner/harness:387:17) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: [],
expected: [ 'load' ],
operator: 'deepStrictEqual',
diff: 'simple'
}
===== mx-M6.log
✖ effective CPUs take the smallest of host, cpuset and every quota level (1.625079ms)
ℹ tests 1
ℹ suites 0
ℹ pass 0
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 55.012035
✖ failing tests:
test at tests/pressure.test.mjs:101:1
✖ effective CPUs take the smallest of host, cpuset and every quota level (1.625079ms)
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
4 !== 1.5
at TestContext.<anonymous> (file:///home/jwoltje/.t3/worktrees/mosaic-stack/morgana-row257/packages/supervisor/tests/pressure.test.mjs:103:10)
at Test.runInAsyncScope (node:async_hooks:226:14)
at Test.run (node:internal/test_runner/test:1402:25)
at Test.start (node:internal/test_runner/test:1262:17)
at startSubtestAfterBootstrap (node:internal/test_runner/harness:387:17) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: 4,
expected: 1.5,
operator: 'strictEqual',
diff: 'simple'
}
===== mx-M7.log
✖ critical stays until reconciliation, then still waits the 30 s release (4.842568ms)
ℹ tests 1
ℹ suites 0
ℹ pass 0
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 54.317924
✖ failing tests:
test at tests/rules.test.mjs:259:1
✖ critical stays until reconciliation, then still waits the 30 s release (4.842568ms)
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
assert.ok(waiting.decisions.every((d) => d.mode === "critical" && d.reasons[0] === "critical-disk"))
at TestContext.<anonymous> (file:///home/jwoltje/.t3/worktrees/mosaic-stack/morgana-row257/packages/supervisor/tests/rules.test.mjs:262:10)
at Test.runInAsyncScope (node:async_hooks:226:14)
at Test.run (node:internal/test_runner/test:1402:25)
at Test.start (node:internal/test_runner/test:1262:17)
at startSubtestAfterBootstrap (node:internal/test_runner/harness:387:17) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '==',
diff: 'simple'
}
===== mx-M8.log
✖ memory under 5% and disk under 1 GiB are critical, at their edges (3.080014ms)
ℹ tests 1
ℹ suites 0
ℹ pass 0
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 49.440823
✖ failing tests:
test at tests/rules.test.mjs:245:1
✖ memory under 5% and disk under 1 GiB are critical, at their edges (3.080014ms)
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
'critical' !== 'held'
at TestContext.<anonymous> (file:///home/jwoltje/.t3/worktrees/mosaic-stack/morgana-row257/packages/supervisor/tests/rules.test.mjs:253:10)
at Test.runInAsyncScope (node:async_hooks:226:14)
at Test.run (node:internal/test_runner/test:1402:25)
at Test.start (node:internal/test_runner/test:1262:17)
at startSubtestAfterBootstrap (node:internal/test_runner/harness:387:17) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: 'critical',
expected: 'held',
operator: 'strictEqual',
diff: 'simple'
}
+42 -1
View File
@@ -101,7 +101,16 @@ export const TASK_PAGE_MAX = 200;
// request allowed past the ordinary 64 KiB envelope (server.mjs and client.mjs), up to its content cap.
export const PROJECT_FILE_VERBS = Object.freeze(['project.files', 'project.file', 'project.file.save']);
// C2-03a: library.sync is the human's fetch, and with `accept` the one way past library-diverged.
export const LIBRARY_VERBS = Object.freeze(['library.file', 'library.status', 'library.sync']);
// Row 224: pure agent lookups are separate from the three human-only verbs. Listing them
// here also makes server.mjs await their host effects instead of serializing a Promise as {}.
const LIBRARY_LOOKUPS = Object.freeze({
'library.find': { method: 'find', fields: ['universe', 'query', 'type', 'book', 'scope', 'offset'], required: ['universe', 'query'] },
'library.get': { method: 'get', fields: ['universe', 'path', 'from', 'links_from'], required: ['universe', 'path'] },
'library.search': { method: 'search', fields: ['universe', 'text', 'book', 'scope', 'offset'], required: ['universe', 'text'] },
'library.manuscript': { method: 'manuscript', fields: ['universe', 'book', 'scope', 'chapter', 'from', 'to'], required: ['universe', 'book'] },
'library.universe': { method: 'universeInfo', fields: ['universe'], required: ['universe'] },
});
export const LIBRARY_VERBS = Object.freeze(['library.file', 'library.status', 'library.sync', ...Object.keys(LIBRARY_LOOKUPS)]);
export const PROJECT_SAVE_VERB = 'project.file.save';
export const PROJECT_SAVE_REQUEST_MAX = 1024 * 1024 + 64 * 1024;
export const WEB_VERBS =Object.freeze(['web.search', 'web.fetch']);
@@ -5352,10 +5361,42 @@ export class Broker {
throw this.#refused(s, ProjectFailure && e instanceof ProjectFailure ? new BusError(e.code) : e, request);
}
}
// Row 224 C2-05b: the agent never supplies a host repo, commit or business. Its current
// holder and business entry are rechecked before/after awaits; runtime owns Git and queries.
async #requestLibraryLookup(s, { verb, args }) {
try {
this.#agent(s);
const business = this.#business(s.business);
const projects = Object.entries(business.projects ?? {}).filter(([, entry]) => entry.kind === 'library');
if (!projects.length || typeof this.#library?.lookup !== 'function') fail('library-not-configured');
const spec = LIBRARY_LOOKUPS[verb];
keys(args, ['project', ...spec.fields], spec.required);
if (!text(args.universe, 160)) fail('invalid-request');
if (args.project === undefined && projects.length !== 1) fail('invalid-request');
const project = args.project ?? projects[0][0];
if (!text(project, 160) || !projects.some(([name]) => name === project)) fail('library-not-configured');
const revision = JSON.stringify(business.projects[project]);
const authorize = () => {
this.#agent(s);
this.#processingGate(s, verb);
if (JSON.stringify(this.#business(s.business).projects?.[project]) !== revision) fail('library-not-configured');
};
const { project: ignored, ...query } = args;
authorize();
const result = await this.#library.lookup({ business, project }, { method: spec.method, args: query, authorize });
authorize();
this.#secretCheck(result);
return result;
} catch (error) {
const Failure = this.#library?.LibraryError;
throw this.#refused(s, Failure && error instanceof Failure ? new BusError(error.code) : error, { verb, args });
}
}
// C2-04 M4: only a verified human reads library text or status. Host Git
// functions stay injected, preserving this module's sandbox import closure.
async #requestLibrary(s, request) {
const { verb, args } = request;
if (Object.hasOwn(LIBRARY_LOOKUPS, verb)) return this.#requestLibraryLookup(s, request);
try {
this.#human(s);
const fields = verb === 'library.file' ? ['project', 'path'] : verb === 'library.sync' ? ['project', 'accept'] : ['project'];
+34
View File
@@ -87,6 +87,38 @@ export async function harnessLevels({ log = (e) => process.stderr.write(`plugins
}
return (name) => levels.get(name) ?? null;
}
// Row 224 C2-05b: a host-only adapter. A failed fetch keeps serving immutable old
// objects, as C2-03a requires; no caller may select a repo, commit or sync --accept.
export function createLibraryLookup({ dataRoot, sync, now = () => Date.now() }) {
const methods = { find: libraryModule.find, get: libraryModule.get, search: libraryModule.search,
manuscript: libraryModule.manuscript, universeInfo: libraryModule.universeInfo };
return async (context, { method, args, authorize }) => {
if (!Object.hasOwn(methods, method)) throw new libraryModule.LibraryError('query-argument-invalid', 'unknown lookup');
authorize();
const source = { ...context, dataRoot };
let state = await libraryModule.libraryStatus(source);
authorize();
// sage-2819 / dewey-0852: failed attempts also pace inline retries. Otherwise an
// outage makes every lookup refetch while lastFetch remains old or null.
const recentAttempt = Math.max(state.lastFetch === null ? -Infinity : Date.parse(state.lastFetch),
state.lastError === null ? -Infinity : Date.parse(state.lastError.at));
const age = now() - recentAttempt;
if (age > libraryModule.SYNC_INTERVAL_MS || age < 0) {
try { await sync(source); }
catch (error) {
// A timer may already be fetching this mirror. Its current served state is still safe.
if (!(error instanceof libraryModule.LibraryError) || error.code !== 'library-sync-busy') throw error;
}
authorize();
state = await libraryModule.libraryStatus(source);
}
if (!state.served) throw new libraryModule.LibraryError('library-commit', 'library has never been synced');
authorize();
return methods[method]({ repo: libraryModule.mirrorPath(source), commit: state.served,
mapping: libraryModule.checkMapping(context.business.projects[context.project]) }, args);
};
}
export async function startBroker({
dataRoot,
businesses,
@@ -310,6 +342,8 @@ export async function startBroker({
store,
businesses,
library: {
lookup: createLibraryLookup({ dataRoot, sync: context =>
processing.work(() => libraryModule.sync(context, syncOptions(context.business))) }),
readFile: context => libraryModule.readLibraryFile({ ...context, dataRoot }),
status: context => libraryModule.libraryStatus({ ...context, dataRoot }),
sync: (context, { accept }) => processing.work(() => libraryModule.sync({ ...context, dataRoot }, { ...syncOptions(context.business), accept })),
+237
View File
@@ -0,0 +1,237 @@
// Row 224: exercise broker authorization, real C2-05a queries and host served snapshots.
// All bodies and remotes are made up. External sync is replaced only at its trusted host seam.
import test from 'node:test';
import assert from 'node:assert/strict';
import { mkdirSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { Store } from '../src/store.mjs';
import { Broker } from '../src/broker.mjs';
import { createLibraryLookup, startBroker } from '../src/runtime.mjs';
import { serve } from '../src/server.mjs';
import { Client } from '../src/client.mjs';
import * as library from '../../library/src/index.mjs';
import { scratchRepo, note } from '../../library/tests/query-fixture.mjs';
import { mappingEntry, git } from '../../library/tests/helpers.mjs';
const NOW = Date.parse('2026-10-22T16:00:00.000Z');
const VERBS = ['library.find', 'library.get', 'library.search', 'library.manuscript', 'library.universe'];
const GET = { universe: 'harbor', path: 'Book One/Characters/Tomas.md' };
const ARGS = [
{ universe: 'harbor', query: 'Tomas' }, GET,
{ universe: 'harbor', text: 'ferry' }, { universe: 'harbor', book: 'Book One' }, { universe: 'harbor' },
];
const TEXTS = {
'Book One/Characters/Tomas.md': note('character', 'Tomas', 'He runs the ferry. [[Elsewhere]]'),
'Book One/Chapters/01.md': '# A ferry\nOne line.\n',
'Book One/Playground/What If.md': note('character', 'Maybe Tomas', 'Unwritten ferry.'),
'Second World/Characters/Elsewhere.md': note('character', 'Elsewhere', 'Other ferry.'),
};
async function world(t, { projects, adapter, claimed = true, now = () => NOW } = {}) {
const f = scratchRepo(t, TEXTS), dataRoot = join(f.root, 'data');
mkdirSync(dataRoot);
const business = { id: 'acme', human: 'jason', vars: { 'bus.tester': true },
projects: projects ?? { books: mappingEntry() }, arbiters: { technical: 'coder', delivery: 'coder' },
roles: { coder: { definition: 'coder', authority: { withinRole: [], crossRole: [] } } } };
const context = { business, project: 'books', dataRoot };
let mirror = null, state = null, fetches = 0;
if (business.projects.books?.kind === 'library') {
mirror = await library.ensureMirror(context);
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', mirror, 'HEAD:refs/heads/served');
state = { version: 1, branch: 'main', served: f.commit, lastFetch: new Date(NOW).toISOString(),
lastError: null, diverged: null, problems: [] };
writeFileSync(join(mirror, library.SYNC_STATE_FILE), JSON.stringify(state), { mode: 0o600 });
}
const setState = patch => {
state = { ...state, ...patch };
writeFileSync(join(mirror, library.SYNC_STATE_FILE), JSON.stringify(state), { mode: 0o600 });
};
const lookup = createLibraryLookup({ dataRoot, now, sync: async () => {
fetches++;
setState({ lastFetch: new Date(now()).toISOString() });
return { state, code: null };
} });
const store = new Store(dataRoot);
t.after(() => store.close());
const broker = new Broker({ store, businesses: { acme: business },
library: adapter ?? { lookup, LibraryError: library.LibraryError } });
const agent = broker.bindLaunch({ business: 'acme', role: 'coder', run: 'r224', harness: 'pi' });
if (claimed) broker.request(agent, { verb: 'role.claim' });
const human = broker.bindHuman({ business: 'acme', human: 'jason', via: 'cli', outsideAgent: true });
const reader = broker.bindReader({ business: 'acme' }), tester = broker.bindTester({ business: 'acme' });
const call = (verb, args = {}, cap = agent) => Promise.resolve().then(() => broker.request(cap, { verb, args }));
return { ...f, dataRoot, business, context, mirror, store, broker, agent, human, reader, tester,
call, lookup, setState, fetches: () => fetches };
}
test('all five agent verbs read the configured served commit through real queries', async t => {
const f = await world(t);
const [find, get, search, manuscript, universe] = await Promise.all(VERBS.map((verb, i) => f.call(verb, ARGS[i])));
for (const result of [find, get, search, manuscript, universe]) assert.equal(result.commit, f.commit);
assert.deepEqual(find.results.map(x => x.path), ['Book One/Characters/Tomas.md']);
assert.match(JSON.stringify(get.lines), /He runs the ferry/);
assert.deepEqual(get.links_out, []); // The only target is in another universe.
assert.ok(search.results.length > 0);
assert.ok(search.results.every(x => !x.path.startsWith('Second World/') && x.scope === 'canon'));
assert.equal(manuscript.path, 'Book One/Chapters/01.md');
assert.deepEqual(universe.books, ['Book One']);
assert.equal(f.fetches(), 0);
const playground = await f.call('library.find', { universe: 'harbor', query: '', scope: 'playground' });
assert.deepEqual(playground.results.map(x => x.path), ['Book One/Playground/What If.md']);
assert.equal(await f.call('library.get', { universe: 'harbor', path: 'Second World/Characters/Elsewhere.md' }), null);
});
test('each verb refuses missing universe and unknown keys before host work', async t => {
let calls = 0;
const f = await world(t, { adapter: { lookup: () => { calls++; } } });
for (const [i, verb] of VERBS.entries()) {
const { universe, ...missing } = ARGS[i];
await assert.rejects(f.call(verb, missing), { code: 'invalid-request' });
for (const extra of [{ repo: '/elsewhere' }, { commit: 'b'.repeat(40) }, { business: 'other' }, { accept: 'b'.repeat(40) }])
await assert.rejects(f.call(verb, { ...ARGS[i], ...extra }), { code: 'invalid-request' });
await assert.rejects(f.call(verb, { ...ARGS[i], universe: '' }), { code: 'invalid-request' });
}
assert.equal(calls, 0);
});
test('no library, missing host adapter and undeclared project refuse library-not-configured', async t => {
const noLibrary = await world(t, { projects: {} });
const noAdapter = await world(t, { adapter: {} });
const f = await world(t);
for (const [i, verb] of VERBS.entries()) {
await assert.rejects(noLibrary.call(verb, ARGS[i]), { code: 'library-not-configured' });
await assert.rejects(noAdapter.call(verb, ARGS[i]), { code: 'library-not-configured' });
await assert.rejects(f.call(verb, { ...ARGS[i], project: 'other' }), { code: 'library-not-configured' });
}
f.business.projects.second = mappingEntry();
f.broker.reloadBusiness('acme', f.business);
await assert.rejects(f.call('library.get', GET), { code: 'invalid-request' });
assert.equal((await f.call('library.get', { ...GET, project: 'books' })).commit, f.commit);
});
test('reads need the current holder on each call; human/tester/reader are not agents', async t => {
const f = await world(t, { claimed: false });
for (const [i, verb] of VERBS.entries()) {
await assert.rejects(f.call(verb, ARGS[i]), { code: 'not-holder' });
await assert.rejects(f.call(verb, ARGS[i], f.human), { code: 'agent-required' });
await assert.rejects(f.call(verb, ARGS[i], f.tester), { code: 'tester-refused' });
await assert.rejects(f.call(verb, ARGS[i], f.reader), { code: 'read-only' });
}
f.broker.request(f.agent, { verb: 'role.claim' });
assert.equal((await f.call('library.get', GET)).commit, f.commit);
f.broker.request(f.agent, { verb: 'role.release' });
await assert.rejects(f.call('library.get', GET), { code: 'not-holder' });
});
test('releasing a claim during an async lookup fences the response', { timeout: 30_000 }, async t => {
let release, entered;
const started = new Promise(resolve => { entered = resolve; });
const f = await world(t, { adapter: { lookup: async () => {
entered(); await new Promise(resolve => { release = resolve; }); return { lines: [[1, 'not returned']] };
} } });
const pending = f.call('library.get', GET);
await started;
f.broker.request(f.agent, { verb: 'role.release' });
release();
await assert.rejects(pending, { code: 'not-holder' });
});
test('a business library removed during an async read cannot leak its result', { timeout: 30_000 }, async t => {
let release, entered;
const started = new Promise(resolve => { entered = resolve; });
const f = await world(t, { adapter: { lookup: async () => {
entered(); await new Promise(resolve => { release = resolve; }); return { lines: [[1, 'not returned']] };
} } });
const pending = f.call('library.get', GET);
await started;
delete f.business.projects.books;
f.broker.reloadBusiness('acme', f.business);
release();
await assert.rejects(pending, { code: 'library-not-configured' });
});
test('freshness boundary, unknown last fetch and failed fetch still use immutable served state', async t => {
const f = await world(t);
f.setState({ lastFetch: new Date(NOW - 60_000).toISOString() });
await f.call('library.get', GET);
assert.equal(f.fetches(), 0);
f.setState({ lastFetch: new Date(NOW - 60_001).toISOString() });
await f.call('library.get', GET);
assert.equal(f.fetches(), 1);
f.setState({ lastFetch: null });
await f.call('library.universe', { universe: 'harbor' });
assert.equal(f.fetches(), 2);
let attempted = 0, clock = NOW;
const failing = createLibraryLookup({ dataRoot: f.dataRoot, now: () => clock, sync: async () => {
attempted++;
f.setState({ lastError: { code: 'library-fetch-failed', at: new Date(clock).toISOString() } });
return { code: 'library-fetch-failed' };
} });
f.setState({ lastFetch: null });
const result = await failing(f.context, { method: 'get', args: GET, authorize() {} });
assert.equal(attempted, 1);
assert.equal(result.commit, f.commit);
for (const elapsed of [1, 59_999, 60_000]) {
clock = NOW + elapsed;
const retry = await failing(f.context, { method: 'get', args: GET, authorize() {} });
assert.equal(retry.commit, f.commit, 'serve the old pin during the failure cooldown');
assert.equal(attempted, 1, 'recent lastError prevents another sync even though lastFetch is null');
}
clock = NOW + 60_001;
assert.equal((await failing(f.context, { method: 'get', args: GET, authorize() {} })).commit, f.commit);
assert.equal(attempted, 2, 'retry only after the failed-attempt interval expires');
});
test('an unreadable sync state fails closed; a busy sync keeps serving the existing pin', async t => {
const f = await world(t);
const lookup = createLibraryLookup({ dataRoot: f.dataRoot, now: () => NOW, sync: async () => {
throw new library.LibraryError('library-sync-busy', 'another fetch');
} });
f.setState({ lastFetch: null });
assert.equal((await lookup(f.context, { method: 'get', args: GET, authorize() {} })).commit, f.commit);
writeFileSync(join(f.mirror, library.SYNC_STATE_FILE), '{bad json', { mode: 0o600 });
await assert.rejects(f.call('library.get', GET), { code: 'library-state-invalid' });
});
test('a stale lookup sees the commit published by its sync, not the prior served commit', async t => {
const f = await world(t);
writeFileSync(join(f.repo, GET.path), note('character', 'Tomas', 'Changed ferry.'));
git(f.repo, 'add', '-A'); git(f.repo, 'commit', '-qm', 'second fixture');
const next = git(f.repo, 'rev-parse', 'HEAD');
const lookup = createLibraryLookup({ dataRoot: f.dataRoot, now: () => NOW, sync: async () => {
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', f.mirror, 'HEAD:refs/heads/served');
f.setState({ served: next, lastFetch: new Date(NOW).toISOString() });
return { code: null };
} });
f.setState({ lastFetch: null });
const result = await lookup(f.context, { method: 'get', args: GET, authorize() {} });
assert.equal(result.commit, next);
assert.match(JSON.stringify(result.lines), /Changed ferry/);
});
test('real socket awaits lookup results and rechecks rejected query arguments', { timeout: 30_000 }, async t => {
const f = await world(t);
const path = join(f.root, 'lookup.sock');
const server = await serve({ broker: f.broker, path }); t.after(() => server.close());
const client = new Client({ path, cap: f.agent });
const result = await client.call('library.get', GET);
assert.equal(result.commit, f.commit);
await assert.rejects(client.call('library.find', { universe: 'harbor', query: '', scope: 'anything' }), /query-argument-invalid/);
});
test('runtime actually injects the lookup adapter, without exposing human library.file', { timeout: 30_000 }, async t => {
const f = scratchRepo(t, TEXTS), dataRoot = join(f.root, 'runtime'); mkdirSync(dataRoot);
const business = { id: 'acme', human: 'jason', plugins: { harness: ['pi'] }, projects: { books: mappingEntry() },
arbiters: { technical: 'coder', delivery: 'coder' },
roles: { coder: { definition: 'coder', authority: { withinRole: [], crossRole: [] } } } };
const runtime = await startBroker({ dataRoot, businesses: { acme: business } }); t.after(() => runtime.close());
const mirror = library.mirrorPath({ business, project: 'books', dataRoot });
git(f.repo, '-c', 'protocol.file.allow=always', 'push', '-q', mirror, 'HEAD:refs/heads/served');
writeFileSync(join(mirror, library.SYNC_STATE_FILE), JSON.stringify({ version: 1, branch: 'main', served: f.commit,
lastFetch: new Date().toISOString(), lastError: null, diverged: null, problems: [] }), { mode: 0o600 });
const cap = runtime.broker.bindLaunch({ business: 'acme', role: 'coder', run: 'r224-runtime', harness: 'pi' });
runtime.broker.request(cap, { verb: 'role.claim' });
assert.equal((await runtime.broker.request(cap, { verb: 'library.get', args: GET })).commit, f.commit);
await assert.rejects(runtime.broker.request(cap, { verb: 'library.file', args: { project: 'books', path: GET.path } }),
{ code: 'human-required' });
});
+1
View File
@@ -528,6 +528,7 @@ export function createLauncher({
contract: readFileSync(resolved.contract, "utf8"),
arbiter: Object.values(b.arbiters).includes(instance),
tracker,
library: Object.values(b.projects ?? {}).some(project => project.kind === "library"),
credentials,
session: { run, launchedBy, workspace: inSandbox, toolSocket, turnMarker, checkouts: inSandbox === workspace ? checkouts : checkouts.map((c) => ({ ...c, path: join(inSandbox, relative(workspace, c.path)) })) },
harness,
+27
View File
@@ -27,6 +27,7 @@ import { bootConfig, loadSystem } from "../src/config.mjs";
import { startTimeOf } from "../src/host.mjs";
import { createLauncher } from "../src/launcher.mjs";
import { fakeHarnesses, fixture, tmp } from "./helpers.mjs";
import { mappingEntry } from "../../library/tests/helpers.mjs";
// Row 264a (#1769): a thread made by hand gets the default session the 3k-3l migration gives a
// legacy thread, with the thread's id.
const DEFAULT_SESSION = [
@@ -533,3 +534,29 @@ test('C2-09a: terminating one creative thread does not signal an unselected peer
await until(() => !readSessions(f.dataRoot).some(row => row.run === runs[0]));
assert.ok(readSessions(f.dataRoot).some(row => row.run === runs[1]));
});
// Row224: the launcher must derive library presence from business projects, not
// from session checkouts (a library never has an agent clone).
test("C2-05b a real launched bundle gets lookups for a library project without a library checkout", { skip, timeout: 60000 }, async t => {
const prepared = prepare(t, { idle: 10, edit: doc => {
doc.projects.books = mappingEntry();
doc.roles.chief.credentials = { gitea: { ...doc.roles.coder.credentials.gitea } };
return doc;
} });
const { f } = prepared;
const host = await start(t, prepared);
const project = host.call("project.create", { name: "library work", repo: "stack" }).id;
const thread = host.call("thread.create", { project, name: "lookup" }).id;
const sent = host.call("thread.send", { thread, body: "inspect bundle" });
// broker.mjs:4854 uses async citation hydration for a business with a library.
const reply = await until(async () => (await host.call("thread.messages", { thread })).find(m => m.in_reply_to === sent.id && m.class === "RESULT"));
assert.equal(reply.from_role, "assistant");
assert.equal(reply.to_role, "human");
const run = launchLog(f).find(e => e.event === "launch" && e.thread === thread).run;
const tools = JSON.parse(readFileSync(join(f.dataRoot, "launches", "acme", run, "bundle", "tools.json"), "utf8"));
assert.deepEqual(tools.filter(tool => tool.name.startsWith("library_")).map(tool => tool.name),
["library_find", "library_get", "library_search", "library_manuscript", "library_universe"]);
const [session] = threadRuns(f, thread);
assert.ok(session);
assert.ok(!session.checkouts?.some(checkout => checkout.project === "books"));
});
+3 -2
View File
@@ -292,6 +292,7 @@ function prompt({ contract, resolved, session, tools, untyped, builtins, persona
// resolved resolveInstance(...) output
// contract the role contract's text
// arbiter true when the business names this instance as an arbiter
// library true when the trusted business configuration has a library project
// tracker true when the broker has task verbs for the business
// credentials the broker's credentialStatus(...) for the instance (metadata only)
// session { run, launchedBy, workspace, toolSocket, turnMarker, checkouts }
@@ -309,11 +310,11 @@ function prompt({ contract, resolved, session, tools, untyped, builtins, persona
// (row 213); an instance with persona.sha256 and no personaDir refuses
// Returns { manifest, files, policy, tools, env } with absolute paths; env is the module's
// adapter variables, which the host writes into session.json.
export function buildBundle({ dir, resolved, contract, arbiter = false, tracker = false, credentials = [], session, harness, route = "primary", routeReason = null, harnessVersion = null, models = null, land = false, skillsDir = null, personaDir = null }) {
export function buildBundle({ dir, resolved, contract, arbiter = false, tracker = false, library = false, credentials = [], session, harness, route = "primary", routeReason = null, harnessVersion = null, models = null, land = false, skillsDir = null, personaDir = null }) {
if (route !== "primary" && route !== "fallback") throw new BundleError(`route ${JSON.stringify(route)} isn't primary or fallback`);
const model = sessionModel(resolved, models, route);
const web = webPlan(resolved, model.harness, harness.builtinWeb());
const tools = typedTools({ resolved, arbiter, tracker, projects: (session.checkouts ?? []).map((c) => c.project), webTyped: !web.builtin, land });
const tools = typedTools({ resolved, arbiter, tracker, library, projects: (session.checkouts ?? []).map((c) => c.project), webTyped: !web.builtin, land });
const untyped = untypedActions(resolved, tools);
const persona = readPersona(resolved, personaDir);
mkdirSync(dir, { recursive: true, mode: 0o700 });
+34 -2
View File
@@ -78,9 +78,10 @@ const TASK_TOOLS = {
const MESSAGE_CLASSES = ["REQUEST", "ASSIGNMENT", "REVIEW-REQUEST", "REVIEW-RESULT", "RESULT", "INFO", "DECISION", "REACTION", "BLOCK"];
// typedTools({ resolved, arbiter, tracker, projects, land }) -> [{ name, verb, action, description, parameters }]
// typedTools({ resolved, arbiter, tracker, projects, land, library }) -> [{ name, verb, action, description, parameters }]
// resolved resolveInstance(...) output for the session's instance
// arbiter true when the business names this instance as an arbiter
// library true when the trusted business configuration has a library project
// tracker true when the broker has task verbs for the business
// projects the project ids the session has a clone of
// land true when the role and thread hold a merge decision a landing could use (landHeld)
@@ -88,7 +89,7 @@ const MESSAGE_CLASSES = ["REQUEST", "ASSIGNMENT", "REVIEW-REQUEST", "REVIEW-RESU
// `verb` is the broker verb; the `launch` tool has verb "launch" and goes
// to the host instead, and "working.push", "working.fetch" and
// "working.land" go to the host's git socket.
export function typedTools({ resolved, arbiter = false, tracker = false, projects = [], webTyped = true, land = false }) {
export function typedTools({ resolved, arbiter = false, tracker = false, projects = [], webTyped = true, land = false, library = false }) {
const held = new Set([...resolved.limits.authority.withinRole, ...resolved.limits.authority.crossRole]);
const tools = [];
const add = (name, verb, action, description, parameters) => tools.push({ name, verb, action, description, parameters });
@@ -222,6 +223,32 @@ export function typedTools({ resolved, arbiter = false, tracker = false, project
url: str("The https url to fetch", { maxLength: 2048 }),
}, ["url"]));
}
// Row224: no library clone or token reaches a session. The broker chooses the served
// commit and requires a universe on every call; cursors page without cutting replies.
if (library) {
const universe = str("Universe ID in the configured library", { minLength: 1 });
const project = str("Library project ID; omit only when the business has one library");
const scope = str("canon by default; playground is opt-in", { enum: ["canon", "playground"] });
const book = str("Book folder from library_universe");
const offset = { type: "integer", minimum: 0, description: "next from the previous page" };
const line = { type: "integer", minimum: 1, description: "One-based line number" };
const base = { universe, project };
add("library_find", "library.find", null, "Find names in one universe, at the served commit. At most 50 results; continue with next as offset.", obj({
...base, query: str("Case-insensitive name or filename substring; empty matches all"),
type: str("Mapped type", { enum: ["character", "place", "research", "tracker", "summary"] }),
book, scope, offset,
}, ["universe", "query"]));
add("library_get", "library.get", null, "Read a mapped note and visible links in one universe at the served commit. next pages lines; links_next pages links. An explicit own-universe playground path is readable.", obj({
...base, path: str("Mapped note path relative to the library"), from: line, links_from: offset,
}, ["universe", "path"]));
add("library_search", "library.search", null, "Search text in one universe at the served commit. At most 50 hits; use next as offset. Playground is opt-in.", obj({
...base, text: str("Nonempty case-insensitive substring", { minLength: 1 }), book, scope, offset,
}, ["universe", "text"]));
add("library_manuscript", "library.manuscript", null, "Read one chapter of a book at the served commit, at most 400 lines. Omit chapter for the first; continue with next or next_chapter.", obj({
...base, book, scope, chapter: str("Chapter filename without extension"), from: line, to: line,
}, ["universe", "book"]));
add("library_universe", "library.universe", null, "Read a universe's mapping at the served commit: books, types, playground prefixes, and hybrid sources/constraints/notes or draft metadata.", obj(base, ["universe"]));
}
add("read_spec", "spec.read", null, "Read a project's committed north-star spec and its file history. commit is an optional full reachable commit ID. The file commit, rather than HEAD, is what a decision cites. confirmed says the file commit was authored by the business's human.", obj({
project: str("Project ID from the business file"),
commit: str("Optional full historical commit ID", { pattern: "^[0-9a-f]{40}$" }),
@@ -277,6 +304,11 @@ export function typedTools({ resolved, arbiter = false, tracker = false, project
if (held.has("task.create"))
add("list_requirements", "requirements", null, "List the approved PRD's requirement ids, each with a one-line title. task_create takes only these. When none fits a request, raise a prd.approve decision instead of creating the task.", obj({}));
}
// mos-1308 / M3: no generic spec, evidence, tracker, decision or write tools. The
// C2-08 web grant still supplies broker web reads to companion and research only.
if (CREATIVE_DEFINITIONS.includes(resolved.definition))
return tools.filter(tool => tool.name === "send_message" || tool.name.startsWith("library_") ||
tool.name === "web_search" || tool.name === "web_fetch");
return tools;
}
@@ -0,0 +1,71 @@
// Row224 M3: exact creative tool sets preserve the existing C2-08 web grants.
import test from 'node:test';
import assert from 'node:assert/strict';
import { mkdirSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import { typedTools } from '../src/tools.mjs';
import { buildBundle } from '../src/bundle.mjs';
import { TYPES } from '../../library/src/mapping.mjs';
import { HARNESS, REPO_SKILLS, resolvedFor, scratch } from './helpers.mjs';
const LOOKUPS = ['library_find', 'library_get', 'library_search', 'library_manuscript', 'library_universe'];
const CREATIVE = ['companion', 'research', 'ideas', 'proofreader', 'fact-checker', 'continuity'];
const role = (t, definition, harness = 'pi') => resolvedFor(t, 'bean', d => {
d.roles.bean = { definition, credentials: { gitea: { ...d.roles.coder.credentials.gitea } }, vars: { harness, model: 'claude-sonnet-5-5', thinking: 'high' } };
});
const expected = definition => ['send_message', ...(['companion', 'research'].includes(definition)
? ['web_search', 'web_fetch'] : []), ...LOOKUPS];
test('M3: per-role snapshots contain six names, or eight with the C2-08 web grant', t => {
for (const definition of CREATIVE) {
const tools = typedTools({ resolved: role(t, definition), library: true, tracker: true, arbiter: true });
assert.deepEqual(tools.map(t => t.name), expected(definition), definition);
assert.ok(tools.every(t => t.parameters.additionalProperties === false));
}
});
test('every noncreative role gets lookups when a library is configured, not otherwise', t => {
for (const instance of ['coder', 'reviewer', 'pm', 'chief']) {
const resolved = resolvedFor(t, instance);
const ordinary = typedTools({ resolved }).map(t => t.name);
assert.ok(!ordinary.some(name => name.startsWith('library_')));
const tools = typedTools({ resolved, library: true });
assert.deepEqual(tools.filter(t => t.name.startsWith('library_')).map(t => t.name), LOOKUPS);
assert.deepEqual(tools.filter(t => !t.name.startsWith('library_')).map(t => t.name), ordinary);
}
});
test('lookup schemas require a universe even for get, include paging, and cannot select repo or commit', t => {
const tools = typedTools({ resolved: role(t, 'ideas'), library: true });
const required = [ ['universe', 'query'], ['universe', 'path'], ['universe', 'text'], ['universe', 'book'], ['universe'] ];
for (const [i, tool] of tools.filter(t => t.name.startsWith('library_')).entries()) {
assert.deepEqual(tool.parameters.required, required[i]);
assert.equal(tool.verb, LOOKUPS[i].replace('_', '.'));
assert.equal(tool.action, null);
assert.equal(tool.parameters.additionalProperties, false);
for (const key of ['repo', 'commit', 'business', 'accept']) assert.equal(Object.hasOwn(tool.parameters.properties, key), false);
}
const byName = Object.fromEntries(tools.map(t => [t.name, t]));
assert.deepEqual(byName.library_find.parameters.properties.scope.enum, ['canon', 'playground']);
assert.deepEqual(byName.library_find.parameters.properties.type.enum, [...TYPES]);
assert.equal(byName.library_find.parameters.properties.offset.minimum, 0);
assert.equal(byName.library_get.parameters.properties.from.minimum, 1);
assert.equal(byName.library_get.parameters.properties.links_from.minimum, 0);
assert.equal(byName.library_manuscript.parameters.properties.chapter.type, 'string');
});
test('bundle flag reaches both harness tool manifests and creative prompt without generic tools', t => {
for (const definition of CREATIVE) for (const harness of ['pi', 'claude-code']) {
const dir = scratch(t), workspace = join(dir, 'ws'); mkdirSync(workspace);
const b = buildBundle({ dir: join(dir, 'bundle'), resolved: role(t, definition, harness), library: true,
contract: '# Creative\n', tracker: true, arbiter: true, skillsDir: REPO_SKILLS,
session: { run: 'r224', launchedBy: 'chief', workspace, toolSocket: join(dir, 't.sock'), turnMarker: join(dir, 'turn.done') },
harness: HARNESS[harness] });
assert.deepEqual(b.tools.map(t => t.name), expected(definition), `${definition} ${harness}`);
const serialized = JSON.parse(readFileSync(b.files.tools, 'utf8'));
assert.deepEqual(serialized.map(t => t.name), expected(definition));
const prompt = readFileSync(b.files.prompt, 'utf8');
assert.match(prompt, /library_find/);
assert.doesNotMatch(prompt, /Bus tools: [^\n]*(read_spec|list_agents|inbox|trail|write_record|raise_decision|list_tasks)/);
}
});
+80
View File
@@ -0,0 +1,80 @@
# Supervisor
Q198 S1 (row 257, #1762): parsers for the host pressure inputs in design D4
(`docs/plans/2026-10-21_q198-system-manager.md`), and a pure evaluator that turns timestamped samples
into a hold or release decision. Nothing here samples the host, runs a process or timer, or enforces
anything. The live sampler, the Console view and admission are S2 and S3.
```sh
node --test packages/supervisor/tests/
```
## Parsers
Each parser takes file text the caller already read. Text that isn't the shape the kernel writes throws
a `SupervisorError` with `kind: "parse"`; a sampler treats that as a missing signal.
| Export | Input | Returns |
|---|---|---|
| `parseLoadavg` | `/proc/loadavg` | `{ load1, load5, load15 }` |
| `parseMeminfo` | `/proc/meminfo` | `{ memTotal, memAvailable, swapTotal, swapFree }` in bytes |
| `parsePsi(text, resource)` | `/proc/pressure/{cpu,memory,io}` or a cgroup `*.pressure` | `{ some, full }`, each `{ avg10, avg60, avg300, total }`; `full` is null only for cpu |
| `parseCpuList` | `/sys/devices/system/cpu/online`, `cpuset.cpus.effective` | how many CPUs the list names |
| `parseCpuMax` | `cpu.max` | the quota in CPUs, or null for `max` |
| `parseMemoryMax` | `memory.max` | bytes, or null for `max` |
| `parseMemoryCurrent` | `memory.current` | bytes |
| `parseMemoryEvents` | `memory.events` | `{ oom, oomKill }` |
| `parseVmstat` | `/proc/vmstat` | `{ pswpin, pswpout, oomKill }`; `oomKill` is the host-only OOM counter, null when absent |
`effectiveCpus({ online, cpuset, cpuMax })` is the smallest of the host count, the cpuset and the quota
at every cgroup level given; a fractional quota stays fractional. `effectiveMemory({ meminfo, levels })`
takes cgroup levels as `{ max, current }` text. The limit is the smallest of `MemTotal` and every numeric
`memory.max`. Available memory is the smallest of `MemAvailable` and `max - current` at each limited level.
`memory.current` counts page cache, so this errs toward holding. With no cgroup input both functions use
the host's figures.
## Evaluator
`evaluate(state, sample, rules, { reconciled })` returns `{ state, decision }`. It starts from
`initialState()`, which is held: no signal has been seen yet. Time is the sample's `at` in milliseconds.
The evaluator never reads the clock.
A sample has these signals: `load { at, load1, cpus }`, `memory { at, available, limit }`,
`psi { at, cpuSome, memoryFull, ioFull }` (avg10 percentages), `oom { at, kills }` and `disk { at, free }`.
| Signal | Hold | Release |
|---|---|---|
| `load1 / cpus` | ≥ 0.80 on 3 consecutive samples | ≤ 0.55 |
| available memory | < max(15% of limit, 1 GiB), on one sample | > max(25% of limit, 2 GiB) |
| PSI avg10 | cpu some ≥ 20, memory full ≥ 2 or io full ≥ 10, on 3 consecutive samples | cpu some < 10, memory full < 0.5 and io full < 3 |
| critical | OOM kill counter rises, memory < 5% of limit, or disk free < 1 GiB | the caller passes `reconciled`, then the normal release |
- **Release** needs every applicable release condition on every sample for 30 s. A dead-band
sample (between hold and release) or a new hold restarts the window.
- **Missing or stale:** a required signal that is missing, malformed, more than 6 s older than the
sample, or later than it holds at once with `health-unknown`. `load`, `memory`, `oom` and `disk` are
always required. PSI becomes required once a sample has carried it. A host without PSI skips the PSI
rows, and one that loses it holds.
- **Consecutive counts** reset after a gap of more than 6 s between samples. The PSI count runs across
resources: any PSI hold condition on 3 consecutive samples holds.
- **OOM:** the first OOM sample is the baseline. A counter that falls (a recreated cgroup) resets the
baseline with the note `oom-counter-reset`.
- **Critical** never ends on thresholds alone. `reconciled` only counts once no critical condition
remains, and the 30 s release still follows. The evaluator never sets it and never touches a kill
switch.
- **A clock that goes backwards** holds with `health-unknown` and the note `clock-backwards`.
`decision` is `{ mode, reasons, unknown, critical, releaseAt, notes }`.
- `mode` is `open`, `held` or `critical`.
- `reasons` are `health-unknown`, `load-high`, `memory-low`, `psi-cpu`, `psi-memory`, `psi-io`,
`hysteresis`, `release-pending` or the critical codes `critical-oom`, `critical-memory` and
`critical-disk`.
- `releaseAt` is set while a release is pending.
`DEFAULT_RULES` is the D4 table, frozen. `resolveRules(overrides)` merges overrides key by key. It refuses
with `kind: "rules"` on an unknown key, a non-number or a contradictory pair, for example:
- a release threshold at or past its hold;
- a release floor at or below its hold floor;
- a critical memory fraction at or above the hold fraction;
- a stale limit at or below the sample interval.
Provider scope (429, quota, Retry-After) is not here; it belongs to S3 and S5.
+11
View File
@@ -0,0 +1,11 @@
{
"name": "@mosaic/supervisor",
"version": "0.1.0",
"private": true,
"description": "Q198 S1: parsers for host pressure inputs and a pure hold and release rules evaluator. No sampler, process or enforcement.",
"license": "UNLICENSED",
"type": "module",
"engines": { "node": ">=24" },
"exports": { ".": "./src/index.mjs" },
"scripts": { "test": "node --test tests/" }
}
+10
View File
@@ -0,0 +1,10 @@
// Every refusal in this package is a SupervisorError. `kind` is "parse" for
// input text that isn't the shape the kernel writes, and "rules" for settings
// the evaluator refuses.
export class SupervisorError extends Error {
constructor(message, kind) {
super(message);
this.name = "SupervisorError";
this.kind = kind;
}
}
+10
View File
@@ -0,0 +1,10 @@
// @mosaic/supervisor: Q198 S1. Parsers for host pressure inputs and a pure
// rules evaluator. Nothing here reads a file, the clock or the environment,
// and nothing enforces: no sampler, process, timer, bus or Console (S2, S3).
export { SupervisorError } from "./errors.mjs";
export {
parseLoadavg, parseMeminfo, parsePsi, parseCpuList, parseCpuMax, parseMemoryMax, parseMemoryCurrent,
parseMemoryEvents, parseVmstat, effectiveCpus, effectiveMemory,
} from "./pressure.mjs";
export { DEFAULT_RULES, resolveRules, initialState, evaluate } from "./rules.mjs";
+165
View File
@@ -0,0 +1,165 @@
// Parsers for the host pressure inputs Q198 D4 names, and the effective CPU
// count and memory limit they give. Every function takes file text the caller
// already read and returns numbers. Nothing here opens a file. Text that isn't
// the shape the kernel writes throws a SupervisorError of kind "parse"; the
// caller turns that into a missing signal.
import { SupervisorError } from "./errors.mjs";
const fail = (what, why) => { throw new SupervisorError(`${what}: ${why}`, "parse"); };
function text(what, value) {
if (typeof value !== "string") fail(what, "expected file text");
return value.endsWith("\n") ? value.slice(0, -1) : value;
}
function integer(what, value) {
if (!/^\d+$/.test(value)) fail(what, `not a non-negative integer: ${JSON.stringify(value)}`);
return Number(value);
}
function decimal(what, value) {
if (!/^\d+(\.\d+)?$/.test(value)) fail(what, `not a non-negative number: ${JSON.stringify(value)}`);
return Number(value);
}
// "key value" lines (memory.events, /proc/vmstat). A repeated key throws.
function keyedIntegers(what, raw) {
const out = new Map();
for (const line of text(what, raw).split("\n")) {
const m = /^([a-z0-9_]+) (\d+)$/.exec(line);
if (!m) fail(what, `bad line ${JSON.stringify(line)}`);
if (out.has(m[1])) fail(what, `repeated key ${m[1]}`);
out.set(m[1], integer(what, m[2]));
}
return out;
}
// /proc/loadavg: "0.52 0.58 0.59 1/1234 56789".
export function parseLoadavg(raw) {
const m = /^(\d+\.\d+) (\d+\.\d+) (\d+\.\d+) (\d+)\/(\d+) (\d+)$/.exec(text("loadavg", raw));
if (!m) fail("loadavg", "not the /proc/loadavg shape");
return { load1: Number(m[1]), load5: Number(m[2]), load15: Number(m[3]) };
}
// /proc/meminfo. Values are returned in bytes; lines without a unit (the
// HugePages counts) are kept as plain counts.
export function parseMeminfo(raw) {
const fields = new Map();
for (const line of text("meminfo", raw).split("\n")) {
const m = /^([A-Za-z0-9_()]+):\s+(\d+)( kB)?$/.exec(line);
if (!m) fail("meminfo", `bad line ${JSON.stringify(line)}`);
if (fields.has(m[1])) fail("meminfo", `repeated key ${m[1]}`);
fields.set(m[1], integer("meminfo", m[2]) * (m[3] ? 1024 : 1));
}
for (const key of ["MemTotal", "MemAvailable"]) if (!fields.has(key)) fail("meminfo", `no ${key}`);
return {
memTotal: fields.get("MemTotal"),
memAvailable: fields.get("MemAvailable"),
swapTotal: fields.get("SwapTotal") ?? null,
swapFree: fields.get("SwapFree") ?? null,
};
}
// /proc/pressure/{cpu,memory,io} or a cgroup's *.pressure file. The cpu file
// may lack the "full" line on older kernels; memory and io must have it.
export function parsePsi(raw, resource) {
if (!["cpu", "memory", "io"].includes(resource)) fail("psi", `unknown resource ${JSON.stringify(resource)}`);
const what = `psi ${resource}`;
const out = { some: null, full: null };
for (const line of text(what, raw).split("\n")) {
const m = /^(some|full) avg10=(\S+) avg60=(\S+) avg300=(\S+) total=(\S+)$/.exec(line);
if (!m) fail(what, `bad line ${JSON.stringify(line)}`);
if (out[m[1]]) fail(what, `repeated ${m[1]} line`);
out[m[1]] = { avg10: decimal(what, m[2]), avg60: decimal(what, m[3]), avg300: decimal(what, m[4]), total: integer(what, m[5]) };
}
if (!out.some) fail(what, "no some line");
if (!out.full && resource !== "cpu") fail(what, "no full line");
return out;
}
// A CPU list such as /sys/devices/system/cpu/online or cpuset.cpus.effective:
// "0-3,6,8-9". Returns how many distinct CPUs it names.
export function parseCpuList(raw) {
const body = text("cpu list", raw);
if (body === "") fail("cpu list", "empty");
const cpus = new Set();
for (const part of body.split(",")) {
const m = /^(\d+)(?:-(\d+))?$/.exec(part);
if (!m) fail("cpu list", `bad range ${JSON.stringify(part)}`);
const lo = integer("cpu list", m[1]);
const hi = m[2] === undefined ? lo : integer("cpu list", m[2]);
if (hi < lo) fail("cpu list", `range ${part} runs backwards`);
for (let c = lo; c <= hi; c++) cpus.add(c);
}
return cpus.size;
}
// cgroup v2 cpu.max: "max 100000" (no quota) or "50000 100000". Returns the
// CPUs the quota allows, or null for no quota.
export function parseCpuMax(raw) {
const m = /^(max|\d+) (\d+)$/.exec(text("cpu.max", raw));
if (!m) fail("cpu.max", "not the cpu.max shape");
const period = integer("cpu.max", m[2]);
if (period === 0) fail("cpu.max", "period is 0");
return m[1] === "max" ? null : integer("cpu.max", m[1]) / period;
}
// cgroup v2 memory.max: "max" (no limit) or bytes. Returns null for "max".
export function parseMemoryMax(raw) {
const body = text("memory.max", raw);
return body === "max" ? null : integer("memory.max", body);
}
// cgroup v2 memory.current: bytes.
export function parseMemoryCurrent(raw) {
return integer("memory.current", text("memory.current", raw));
}
// cgroup v2 memory.events. oom_kill is the counter the critical row watches.
export function parseMemoryEvents(raw) {
const fields = keyedIntegers("memory.events", raw);
if (!fields.has("oom_kill")) fail("memory.events", "no oom_kill");
return { oom: fields.get("oom") ?? null, oomKill: fields.get("oom_kill") };
}
// /proc/vmstat. Swap activity is reported only (D4 has no threshold for it).
// oom_kill is the host-only OOM counter; it is null on a kernel without it.
export function parseVmstat(raw) {
const fields = keyedIntegers("vmstat", raw);
for (const key of ["pswpin", "pswpout"]) if (!fields.has(key)) fail("vmstat", `no ${key}`);
return { pswpin: fields.get("pswpin"), pswpout: fields.get("pswpout"), oomKill: fields.get("oom_kill") ?? null };
}
// Effective CPUs: the smallest of the host's online CPUs, the cpuset and the
// quota at every cgroup level given. With no cgroup input it is the host count.
// A fractional quota stays fractional.
export function effectiveCpus({ online, cpuset = null, cpuMax = [] }) {
let cpus = parseCpuList(online);
if (cpuset !== null) cpus = Math.min(cpus, parseCpuList(cpuset));
for (const level of cpuMax) {
const quota = parseCpuMax(level);
if (quota !== null) cpus = Math.min(cpus, quota);
}
return cpus;
}
// Effective memory. `levels` lists the cgroup levels that apply, each
// { max, current } as file text. The limit is the smallest of MemTotal and every
// numeric memory.max. Available memory is the smallest of MemAvailable and
// (max - current) at every limited level. memory.current counts page cache,
// so this errs toward holding. With no levels it is the host's figures.
export function effectiveMemory({ meminfo, levels = [] }) {
const host = parseMeminfo(meminfo);
let limit = host.memTotal;
let available = host.memAvailable;
for (const level of levels) {
const max = parseMemoryMax(level.max);
if (max === null) continue;
if (level.current === undefined || level.current === null) fail("memory.current", "missing for a limited level");
const current = parseMemoryCurrent(level.current);
limit = Math.min(limit, max);
available = Math.min(available, Math.max(0, max - current));
}
return { limit, available };
}
+201
View File
@@ -0,0 +1,201 @@
// The Q198 D4 hold and release rules as a pure evaluator. evaluate() takes the
// previous state, one timestamped sample and the rules, and returns the next
// state and a decision. It never reads the clock, a file or the environment:
// time is the sample's `at`, in milliseconds. Nothing here enforces anything;
// the decision is advice for S2 and S3.
import { SupervisorError } from "./errors.mjs";
const GiB = 1024 ** 3;
// The D4 table. Defaults are the design's proposals, not measured limits.
export const DEFAULT_RULES = deepFreeze({
sampleMs: 2000,
staleMs: 6000,
releaseMs: 30000,
load: { hold: 0.8, release: 0.55, samples: 3 },
memory: { holdFraction: 0.15, holdBytes: 1 * GiB, releaseFraction: 0.25, releaseBytes: 2 * GiB },
psi: {
samples: 3,
hold: { cpuSome: 20, memoryFull: 2, ioFull: 10 },
release: { cpuSome: 10, memoryFull: 0.5, ioFull: 3 },
},
critical: { memoryFraction: 0.05, diskBytes: 1 * GiB },
});
function deepFreeze(o) {
for (const v of Object.values(o)) if (v && typeof v === "object") deepFreeze(v);
return Object.freeze(o);
}
const refuse = (why) => { throw new SupervisorError(`rules: ${why}`, "rules"); };
// Overrides merge onto the defaults key by key. An unknown key, a value of the
// wrong type or a contradictory pair refuses.
function merge(base, over, at) {
if (over === undefined) return structuredClone(base);
if (!over || typeof over !== "object" || Array.isArray(over)) refuse(`${at || "rules"} must be an object`);
const out = {};
for (const key of Object.keys(over)) if (!Object.hasOwn(base, key)) refuse(`unknown key ${at ? `${at}.` : ""}${key}`);
for (const [key, value] of Object.entries(base)) {
const name = at ? `${at}.${key}` : key;
if (typeof value === "object") out[key] = merge(value, over[key], name);
else {
const v = over[key] ?? value;
if (typeof v !== "number" || !Number.isFinite(v)) refuse(`${name} must be a finite number`);
out[key] = v;
}
}
return out;
}
export function resolveRules(overrides = {}) {
const r = merge(DEFAULT_RULES, overrides, "");
const check = (ok, why) => { if (!ok) refuse(why); };
const fraction = (v) => v > 0 && v < 1;
const percent = (v) => v >= 0 && v <= 100;
check(r.sampleMs > 0, "sampleMs must be above 0");
check(r.staleMs > r.sampleMs, "staleMs must be above sampleMs");
check(r.releaseMs >= 0, "releaseMs must not be negative");
for (const k of ["load", "psi"]) check(Number.isInteger(r[k].samples) && r[k].samples >= 1, `${k}.samples must be an integer of at least 1`);
check(r.load.release > 0 && r.load.release < r.load.hold, "load.release must be above 0 and below load.hold");
check(fraction(r.memory.holdFraction) && fraction(r.memory.releaseFraction), "memory fractions must be between 0 and 1");
check(r.memory.releaseFraction > r.memory.holdFraction, "memory.releaseFraction must be above memory.holdFraction");
check(r.memory.holdBytes >= 0 && r.memory.releaseBytes > r.memory.holdBytes, "memory.releaseBytes must be above memory.holdBytes");
for (const k of ["cpuSome", "memoryFull", "ioFull"]) {
check(percent(r.psi.hold[k]) && percent(r.psi.release[k]), `psi ${k} values must be percentages`);
check(r.psi.release[k] > 0 && r.psi.release[k] < r.psi.hold[k], `psi.release.${k} must be above 0 and below psi.hold.${k}`);
}
check(fraction(r.critical.memoryFraction) && r.critical.memoryFraction < r.memory.holdFraction,
"critical.memoryFraction must be between 0 and memory.holdFraction");
check(r.critical.diskBytes > 0, "critical.diskBytes must be above 0");
return deepFreeze(r);
}
// The state before any sample: held, because no signal has been seen yet.
export function initialState() {
return { mode: "held", lastAt: null, loadCount: 0, psiCount: 0, psiSeen: false, oomBaseline: null, releaseSince: null, critical: [] };
}
const SIGNAL_FIELDS = {
load: ["load1", "cpus"],
memory: ["available", "limit"],
psi: ["cpuSome", "memoryFull", "ioFull"],
oom: ["kills"],
disk: ["free"],
};
// A signal is usable when every field is a non-negative finite number, cpus
// and limit are above 0, and its own timestamp is no later than the sample and
// no more than staleMs before it.
function fresh(name, signal, now, rules) {
if (!signal || typeof signal !== "object") return false;
if (typeof signal.at !== "number" || !Number.isFinite(signal.at)) return false;
if (signal.at > now || now - signal.at > rules.staleMs) return false;
for (const f of SIGNAL_FIELDS[name]) if (typeof signal[f] !== "number" || !Number.isFinite(signal[f]) || signal[f] < 0) return false;
if (name === "load" && signal.cpus <= 0) return false;
if (name === "memory" && signal.limit <= 0) return false;
return true;
}
// Sample: { at, load: { at, load1, cpus }, memory: { at, available, limit },
// psi: { at, cpuSome, memoryFull, ioFull } (avg10 percentages),
// oom: { at, kills }, disk: { at, free } }. `reconciled` is the caller's word
// that reconciliation finished; only it lets a critical hold end.
export function evaluate(state, sample, rules = DEFAULT_RULES, { reconciled = false } = {}) {
if (!sample || typeof sample.at !== "number" || !Number.isFinite(sample.at)) {
throw new SupervisorError("sample: at must be a finite number of milliseconds", "sample");
}
const now = sample.at;
const next = { ...state, critical: [...state.critical], lastAt: now };
const unknown = [];
const notes = [];
const gap = state.lastAt === null ? Infinity : now - state.lastAt;
// A count of consecutive samples survives only a gap of at most staleMs.
const contiguous = gap >= 0 && gap <= rules.staleMs;
if (gap < 0) { unknown.push("clock"); notes.push("clock-backwards"); }
let loadHold = false, loadReleases = false;
if (fresh("load", sample.load, now, rules)) {
const ratio = sample.load.load1 / sample.load.cpus;
next.loadCount = ratio >= rules.load.hold ? (contiguous ? state.loadCount : 0) + 1 : 0;
loadHold = next.loadCount >= rules.load.samples;
loadReleases = ratio <= rules.load.release;
} else { unknown.push("load"); next.loadCount = 0; }
let memoryLow = false, memoryReleases = false, criticalMemory = false;
if (fresh("memory", sample.memory, now, rules)) {
const { available, limit } = sample.memory;
const m = rules.memory;
// "Whichever demands more free memory": the larger floor applies.
memoryLow = available < Math.max(m.holdFraction * limit, m.holdBytes);
memoryReleases = available > Math.max(m.releaseFraction * limit, m.releaseBytes);
criticalMemory = available < rules.critical.memoryFraction * limit;
} else unknown.push("memory");
// PSI is required once it has been seen. A host that never had it skips the
// PSI rows; one that loses it holds with health-unknown.
let psiHold = false, psiReleases = true;
const psiOver = [];
if (sample.psi !== undefined && sample.psi !== null) next.psiSeen = true;
if (fresh("psi", sample.psi, now, rules)) {
const p = sample.psi, h = rules.psi.hold, r = rules.psi.release;
if (p.cpuSome >= h.cpuSome) psiOver.push("psi-cpu");
if (p.memoryFull >= h.memoryFull) psiOver.push("psi-memory");
if (p.ioFull >= h.ioFull) psiOver.push("psi-io");
next.psiCount = psiOver.length ? (contiguous ? state.psiCount : 0) + 1 : 0;
psiHold = next.psiCount >= rules.psi.samples;
psiReleases = p.cpuSome < r.cpuSome && p.memoryFull < r.memoryFull && p.ioFull < r.ioFull;
} else {
next.psiCount = 0;
if (next.psiSeen) { unknown.push("psi"); psiReleases = false; }
}
let criticalOom = false;
if (fresh("oom", sample.oom, now, rules)) {
const kills = sample.oom.kills;
if (state.oomBaseline !== null && kills > state.oomBaseline) criticalOom = true;
if (state.oomBaseline !== null && kills < state.oomBaseline) notes.push("oom-counter-reset");
next.oomBaseline = kills;
} else unknown.push("oom");
let criticalDisk = false;
if (fresh("disk", sample.disk, now, rules)) criticalDisk = sample.disk.free < rules.critical.diskBytes;
else unknown.push("disk");
const criticalNow = [];
if (criticalOom) criticalNow.push("critical-oom");
if (criticalMemory) criticalNow.push("critical-memory");
if (criticalDisk) criticalNow.push("critical-disk");
const decide = (mode, reasons, releaseAt = null) => {
next.mode = mode;
return { state: next, decision: { mode, reasons, unknown, critical: [...next.critical], releaseAt, notes } };
};
if (criticalNow.length) {
for (const c of criticalNow) if (!next.critical.includes(c)) next.critical.push(c);
next.releaseSince = null;
return decide("critical", [...next.critical]);
}
// Critical ends only on the caller's reconciliation, never on thresholds alone.
if (state.mode === "critical" && !reconciled) {
next.releaseSince = null;
return decide("critical", [...next.critical]);
}
const mode = state.mode === "critical" ? "held" : state.mode;
next.critical = [];
const reasons = [];
if (unknown.length) reasons.push("health-unknown");
if (loadHold) reasons.push("load-high");
if (memoryLow) reasons.push("memory-low");
if (psiHold) reasons.push(...psiOver);
if (reasons.length) { next.releaseSince = null; return decide("held", reasons); }
if (mode === "open") { next.releaseSince = null; return decide("open", []); }
if (!(loadReleases && memoryReleases && psiReleases)) { next.releaseSince = null; return decide("held", ["hysteresis"]); }
next.releaseSince = state.releaseSince !== null && contiguous ? state.releaseSince : now;
if (now - next.releaseSince >= rules.releaseMs) { next.releaseSince = null; return decide("open", []); }
return decide("held", ["release-pending"], next.releaseSince + rules.releaseMs);
}
+133
View File
@@ -0,0 +1,133 @@
// Parser and effective-limit tests. Every input is fixture text shaped like
// the kernel's files; nothing reads /proc, /sys or a cgroup.
import { test } from "node:test";
import assert from "node:assert/strict";
import {
SupervisorError, parseLoadavg, parseMeminfo, parsePsi, parseCpuList, parseCpuMax, parseMemoryMax,
parseMemoryCurrent, parseMemoryEvents, parseVmstat, effectiveCpus, effectiveMemory,
} from "../src/index.mjs";
const GiB = 1024 ** 3;
const isParse = (re) => (e) => e instanceof SupervisorError && e.kind === "parse" && re.test(e.message);
const MEMINFO = [
"MemTotal: 32768000 kB",
"MemFree: 1024000 kB",
"MemAvailable: 8192000 kB",
"SwapTotal: 4096000 kB",
"SwapFree: 4000000 kB",
"HugePages_Total: 0",
"Hugepagesize: 2048 kB",
"",
].join("\n");
test("loadavg parses the three loads", () => {
assert.deepEqual(parseLoadavg("0.52 1.58 12.00 3/1234 56789\n"), { load1: 0.52, load5: 1.58, load15: 12 });
});
test("a malformed loadavg throws", () => {
for (const bad of ["", "0.52 0.58 0.59\n", "0.52 0.58 0.59 1/2\n", "x 0.58 0.59 1/2 3\n", "0.52 0.58 0.59 1/2 3 4\n", 42]) {
assert.throws(() => parseLoadavg(bad), isParse(/loadavg/), JSON.stringify(bad));
}
});
test("meminfo returns bytes and keeps unitless counts", () => {
assert.deepEqual(parseMeminfo(MEMINFO), {
memTotal: 32768000 * 1024, memAvailable: 8192000 * 1024, swapTotal: 4096000 * 1024, swapFree: 4000000 * 1024,
});
});
test("meminfo without MemAvailable, with a repeated key or a bad line throws", () => {
assert.throws(() => parseMeminfo("MemTotal: 1 kB\n"), isParse(/no MemAvailable/));
assert.throws(() => parseMeminfo(MEMINFO + "MemTotal: 1 kB\n"), isParse(/repeated key MemTotal/));
assert.throws(() => parseMeminfo(MEMINFO + "Broken line\n"), isParse(/bad line/));
assert.throws(() => parseMeminfo(MEMINFO.replace("8192000", "-1")), isParse(/bad line/));
});
test("PSI parses some and full, and cpu may lack full", () => {
const mem = "some avg10=1.50 avg60=0.20 avg300=0.00 total=123\nfull avg10=0.75 avg60=0.10 avg300=0.00 total=45\n";
assert.deepEqual(parsePsi(mem, "memory"), {
some: { avg10: 1.5, avg60: 0.2, avg300: 0, total: 123 },
full: { avg10: 0.75, avg60: 0.1, avg300: 0, total: 45 },
});
assert.equal(parsePsi("some avg10=20.00 avg60=0.00 avg300=0.00 total=9\n", "cpu").full, null);
});
test("PSI without a full line for memory or io, or with a bad line, throws", () => {
const someOnly = "some avg10=0.00 avg60=0.00 avg300=0.00 total=0\n";
assert.throws(() => parsePsi(someOnly, "memory"), isParse(/no full line/));
assert.throws(() => parsePsi(someOnly, "io"), isParse(/no full line/));
assert.throws(() => parsePsi("full avg10=0.00 avg60=0.00 avg300=0.00 total=0\n", "cpu"), isParse(/no some line/));
assert.throws(() => parsePsi(someOnly + someOnly, "cpu"), isParse(/repeated some/));
assert.throws(() => parsePsi("some avg10=x avg60=0.00 avg300=0.00 total=0\n", "cpu"), isParse(/not a non-negative number/));
assert.throws(() => parsePsi(someOnly, "gpu"), isParse(/unknown resource/));
});
test("CPU lists count distinct CPUs", () => {
assert.equal(parseCpuList("0-3,6,8-9\n"), 7);
assert.equal(parseCpuList("0\n"), 1);
assert.equal(parseCpuList("0-3,2-5"), 6);
for (const bad of ["", "\n", "3-1", "0,,1", "a-b", "0-"]) assert.throws(() => parseCpuList(bad), isParse(/cpu list/), bad);
});
test("cpu.max gives the quota in CPUs, or null for max", () => {
assert.equal(parseCpuMax("max 100000\n"), null);
assert.equal(parseCpuMax("50000 100000\n"), 0.5);
assert.equal(parseCpuMax("200000 100000\n"), 2);
for (const bad of ["max", "50000 0", "-1 100000", "max max"]) assert.throws(() => parseCpuMax(bad), isParse(/cpu.max/), bad);
});
test("memory.max, memory.current and memory.events", () => {
assert.equal(parseMemoryMax("max\n"), null);
assert.equal(parseMemoryMax("4294967296\n"), 4 * GiB);
assert.throws(() => parseMemoryMax("4G\n"), isParse(/memory.max/));
assert.equal(parseMemoryCurrent("123456\n"), 123456);
assert.throws(() => parseMemoryCurrent(""), isParse(/memory.current/));
assert.deepEqual(parseMemoryEvents("low 0\nhigh 4\nmax 2\noom 1\noom_kill 1\noom_group_kill 0\n"), { oom: 1, oomKill: 1 });
assert.throws(() => parseMemoryEvents("low 0\noom 1\n"), isParse(/no oom_kill/));
assert.throws(() => parseMemoryEvents("oom_kill 1\noom_kill 2\n"), isParse(/repeated key/));
});
test("vmstat gives swap counters and the host OOM counter", () => {
assert.deepEqual(parseVmstat("nr_free_pages 100\npswpin 7\npswpout 9\noom_kill 3\n"), { pswpin: 7, pswpout: 9, oomKill: 3 });
assert.deepEqual(parseVmstat("pswpin 0\npswpout 0\n"), { pswpin: 0, pswpout: 0, oomKill: null });
assert.throws(() => parseVmstat("pswpin 0\n"), isParse(/no pswpout/));
});
test("effective CPUs fall back to the host count with no cgroup input", () => {
assert.equal(effectiveCpus({ online: "0-15\n" }), 16);
});
test("effective CPUs take the smallest of host, cpuset and every quota level", () => {
assert.equal(effectiveCpus({ online: "0-15\n", cpuset: "0-3\n" }), 4);
assert.equal(effectiveCpus({ online: "0-15\n", cpuset: "0-3\n", cpuMax: ["max 100000\n", "150000 100000\n"] }), 1.5);
// The outer level is the tighter one.
assert.equal(effectiveCpus({ online: "0-15\n", cpuMax: ["800000 100000\n", "200000 100000\n"] }), 2);
// A quota above the host count doesn't raise it.
assert.equal(effectiveCpus({ online: "0-1\n", cpuMax: ["800000 100000\n"] }), 2);
assert.equal(effectiveCpus({ online: "0-15\n", cpuMax: ["max 100000\n"] }), 16);
});
test("effective memory falls back to the host with no cgroup levels, or only max", () => {
const host = { limit: 32768000 * 1024, available: 8192000 * 1024 };
assert.deepEqual(effectiveMemory({ meminfo: MEMINFO }), host);
assert.deepEqual(effectiveMemory({ meminfo: MEMINFO, levels: [{ max: "max\n" }] }), host);
});
test("effective memory uses the tightest limited level", () => {
// 4 GiB limit with 3 GiB used: 1 GiB available, below the host's MemAvailable.
assert.deepEqual(effectiveMemory({ meminfo: MEMINFO, levels: [{ max: `${4 * GiB}\n`, current: `${3 * GiB}\n` }] }),
{ limit: 4 * GiB, available: 1 * GiB });
// Nested: the inner level is loose, the outer one is nearly full.
assert.deepEqual(effectiveMemory({ meminfo: MEMINFO, levels: [
{ max: "max\n" }, { max: `${16 * GiB}\n`, current: `${2 * GiB}\n` }, { max: `${6 * GiB}\n`, current: `${5.5 * GiB}\n` },
] }), { limit: 6 * GiB, available: 0.5 * GiB });
// Usage above the limit floors at 0.
assert.equal(effectiveMemory({ meminfo: MEMINFO, levels: [{ max: "100\n", current: "200\n" }] }).available, 0);
// A limit above MemTotal doesn't raise the limit.
assert.equal(effectiveMemory({ meminfo: MEMINFO, levels: [{ max: `${64 * GiB}\n`, current: "0\n" }] }).limit, 32768000 * 1024);
});
test("a limited level without memory.current throws", () => {
assert.throws(() => effectiveMemory({ meminfo: MEMINFO, levels: [{ max: `${4 * GiB}\n` }] }), isParse(/missing for a limited level/));
});
+28
View File
@@ -0,0 +1,28 @@
// S1 has no live enforcement: the source must not reach the filesystem, a
// process, the network or the clock. This reads the package's own source text.
import { test } from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import path from "node:path";
const SRC = path.join(import.meta.dirname, "..", "src");
const FORBIDDEN = [
/\bfrom\s+["'](node:)?(fs|fs\/promises|child_process|net|http|https|dgram|tls|worker_threads|os)["']/,
/\bimport\s*\(/,
/\brequire\s*\(/,
/\bprocess\./,
/\bDate\.now\b/,
/\bnew Date\b/,
/\bperformance\.now\b/,
/\bset(Timeout|Interval|Immediate)\b/,
];
test("src reads no file, process, network or clock", () => {
const files = fs.readdirSync(SRC).filter((f) => f.endsWith(".mjs"));
assert.deepEqual(files.sort(), ["errors.mjs", "index.mjs", "pressure.mjs", "rules.mjs"]);
for (const f of files) {
const body = fs.readFileSync(path.join(SRC, f), "utf8");
for (const re of FORBIDDEN) assert.doesNotMatch(body, re, `${f} matches ${re}`);
for (const m of body.matchAll(/\bfrom\s+["']([^"']+)["']/g)) assert.match(m[1], /^\.\/[a-z]+\.mjs$/, `${f} imports ${m[1]}`);
}
});
+339
View File
@@ -0,0 +1,339 @@
// Evaluator tests on a fake clock: each sample's `at` is the time. Synthetic
// numbers only; nothing samples the host or generates load.
import { test } from "node:test";
import assert from "node:assert/strict";
import { SupervisorError, DEFAULT_RULES, resolveRules, initialState, evaluate } from "../src/index.mjs";
const GiB = 1024 ** 3;
const LIMIT = 32 * GiB;
const R = DEFAULT_RULES;
// A healthy sample at `at`. `patch` changes fields per signal; null drops one.
function sample(at, patch = {}) {
const base = {
load: { load1: 0.1, cpus: 1 },
memory: { available: 16 * GiB, limit: LIMIT },
psi: { cpuSome: 0, memoryFull: 0, ioFull: 0 },
oom: { kills: 0 },
disk: { free: 100 * GiB },
};
const out = { at };
for (const [name, fields] of Object.entries(base)) {
if (patch[name] === null) continue;
out[name] = { at, ...fields, ...patch[name] };
}
return out;
}
// Fold samples through evaluate; returns every decision and the last state.
function run(state, samples, rules = R, opts) {
const decisions = [];
for (const s of samples) {
const r = evaluate(state, s, rules, typeof opts === "function" ? opts(s) : opts);
state = r.state;
decisions.push(r.decision);
}
return { state, decisions, last: decisions.at(-1) };
}
// Samples every 2 s from `from` to `to` inclusive.
function every2s(from, to, patch) {
const out = [];
for (let at = from; at <= to; at += 2000) out.push(sample(at, typeof patch === "function" ? patch(at) : patch));
return out;
}
// An open state whose last sample was at 30000.
function opened() {
const { state, last } = run(initialState(), every2s(0, 30000));
assert.equal(last.mode, "open");
return state;
}
// A state held by a load hold, last sample at 36000.
function heldByLoad() {
const { state, last } = run(opened(), every2s(32000, 36000, { load: { load1: 0.9 } }));
assert.deepEqual(last.reasons, ["load-high"]);
return state;
}
const modes = (r) => r.decisions.map((d) => d.mode);
test("the first state is held, and healthy samples open it after exactly 30 s", () => {
assert.equal(initialState().mode, "held");
const first = evaluate(initialState(), sample(0));
assert.equal(first.decision.mode, "held");
assert.deepEqual(first.decision.reasons, ["release-pending"]);
assert.equal(first.decision.releaseAt, 30000);
const r = run(initialState(), [sample(0), sample(2000), sample(6000), sample(10000), sample(14000), sample(18000),
sample(22000), sample(26000), sample(29999), sample(30000)]);
assert.deepEqual(r.decisions.at(-2).reasons, ["release-pending"]);
assert.equal(r.decisions.at(-2).mode, "held");
assert.equal(r.last.mode, "open");
});
test("load holds at 0.80 for 3 samples and not at 0.7999", () => {
const at = run(opened(), every2s(32000, 36000, { load: { load1: 0.8 } }));
assert.deepEqual(modes(at), ["open", "open", "held"]);
assert.deepEqual(at.last.reasons, ["load-high"]);
const below = run(opened(), every2s(32000, 50000, { load: { load1: 0.7999 } }));
assert.ok(below.decisions.every((d) => d.mode === "open"));
});
test("load divides by effective CPUs, fractional ones included", () => {
const r = run(opened(), every2s(32000, 36000, { load: { load1: 0.4, cpus: 0.5 } }));
assert.equal(r.last.mode, "held");
const r2 = run(opened(), every2s(32000, 36000, { load: { load1: 3.1, cpus: 4 } }));
assert.equal(r2.last.mode, "open");
});
test("a gap of more than 6 s resets the consecutive count", () => {
const high = { load: { load1: 0.9 } };
const r = run(opened(), [sample(32000, high), sample(34000, high), sample(40001, high), sample(42001, high), sample(44001, high)]);
assert.deepEqual(modes(r), ["open", "open", "open", "open", "held"]);
const ok = run(opened(), [sample(32000, high), sample(34000, high), sample(40000, high)]);
assert.equal(ok.last.mode, "held");
});
test("load releases at 0.55 and not at 0.5501", () => {
const dead = run(heldByLoad(), every2s(38000, 80000, { load: { load1: 0.5501 } }));
assert.ok(dead.decisions.every((d) => d.mode === "held" && d.reasons[0] === "hysteresis"));
const r = run(heldByLoad(), every2s(38000, 68000, { load: { load1: 0.55 } }));
assert.equal(r.decisions.at(-2).mode, "held");
assert.equal(r.last.mode, "open");
});
test("a dead-band sample restarts the 30 s release window", () => {
const r = run(heldByLoad(), [
...every2s(38000, 60000),
sample(62000, { load: { load1: 0.6 } }),
...every2s(64000, 92000),
sample(94000),
]);
const at = (t) => r.decisions[r.decisions.length - 1 - (94000 - t) / 2000];
assert.equal(at(62000).reasons[0], "hysteresis");
assert.equal(at(92000).mode, "held");
assert.equal(at(94000).mode, "open");
});
test("memory holds below the larger floor, on one sample", () => {
const floor = Math.max(R.memory.holdFraction * LIMIT, R.memory.holdBytes);
assert.ok(floor > R.memory.holdBytes); // 15% of 32 GiB is the larger floor here
const just = Math.ceil(floor);
assert.equal(evaluate(opened(), sample(32000, { memory: { available: just } })).decision.mode, "open");
const low = evaluate(opened(), sample(32000, { memory: { available: just - 1 } })).decision;
assert.equal(low.mode, "held");
assert.deepEqual(low.reasons, ["memory-low"]);
});
test("on a small limit the 1 GiB floor is the larger one", () => {
const small = 4 * GiB; // 15% is 0.6 GiB, so 1 GiB applies
const s = (available) => sample(32000, { memory: { available, limit: small } });
assert.equal(evaluate(opened(), s(1 * GiB)).decision.mode, "open");
assert.deepEqual(evaluate(opened(), s(1 * GiB - 1)).decision.reasons, ["memory-low"]);
});
test("memory releases only above the larger release floor", () => {
// 3 GiB is below the 15% floor (4.8 GiB) and above the 5% critical line (1.6 GiB).
const held = evaluate(opened(), sample(32000, { memory: { available: 3 * GiB } })).state;
assert.equal(held.mode, "held");
const rel = Math.max(R.memory.releaseFraction * LIMIT, R.memory.releaseBytes);
const stay = run(held, every2s(34000, 80000, { memory: { available: Math.floor(rel) } }));
assert.ok(stay.decisions.every((d) => d.mode === "held"));
const go = run(held, every2s(34000, 64000, { memory: { available: Math.floor(rel) + 1 } }));
assert.equal(go.last.mode, "open");
const small = { limit: 4 * GiB }; // 25% is 1 GiB, so 2 GiB applies
const stay2 = run(held, every2s(34000, 80000, { memory: { ...small, available: 2 * GiB } }));
assert.ok(stay2.decisions.every((d) => d.mode === "held"));
const go2 = run(held, every2s(34000, 64000, { memory: { ...small, available: 2 * GiB + 1 } }));
assert.equal(go2.last.mode, "open");
});
test("each PSI hold threshold holds at its value for 3 samples and not just below", () => {
for (const [field, code] of [["cpuSome", "psi-cpu"], ["memoryFull", "psi-memory"], ["ioFull", "psi-io"]]) {
const at = run(opened(), every2s(32000, 36000, { psi: { [field]: R.psi.hold[field] } }));
assert.deepEqual(modes(at), ["open", "open", "held"], field);
assert.deepEqual(at.last.reasons, [code], field);
const below = run(opened(), every2s(32000, 50000, { psi: { [field]: R.psi.hold[field] - 0.01 } }));
assert.ok(below.decisions.every((d) => d.mode === "open"), field);
}
});
test("the PSI count runs across resources", () => {
const r = run(opened(), [sample(32000, { psi: { cpuSome: 25 } }), sample(34000, { psi: { memoryFull: 3 } }), sample(36000, { psi: { ioFull: 12 } })]);
assert.deepEqual(modes(r), ["open", "open", "held"]);
assert.deepEqual(r.last.reasons, ["psi-io"]);
});
test("each PSI release threshold blocks at its value and releases just below", () => {
const held = run(opened(), every2s(32000, 36000, { psi: { cpuSome: 30 } })).state;
for (const field of ["cpuSome", "memoryFull", "ioFull"]) {
const stay = run(held, every2s(38000, 80000, { psi: { [field]: R.psi.release[field] } }));
assert.ok(stay.decisions.every((d) => d.mode === "held" && d.reasons[0] === "hysteresis"), field);
const go = run(held, every2s(38000, 68000, { psi: { [field]: R.psi.release[field] - 0.01 } }));
assert.equal(go.last.mode, "open", field);
}
});
test("a missing required signal holds at once with health-unknown", () => {
for (const name of ["load", "memory", "oom", "disk"]) {
const d = evaluate(opened(), sample(32000, { [name]: null })).decision;
assert.equal(d.mode, "held", name);
assert.deepEqual(d.reasons, ["health-unknown"], name);
assert.deepEqual(d.unknown, [name], name);
}
});
test("a signal is stale at more than 6 s old and fresh at exactly 6 s", () => {
const stale = { at: 32000, load1: 0.1, cpus: 1 };
const s = (signalAt) => ({ ...sample(32000), load: { ...stale, at: signalAt } });
assert.equal(evaluate(opened(), s(26000)).decision.mode, "open");
assert.deepEqual(evaluate(opened(), s(25999)).decision.unknown, ["load"]);
assert.deepEqual(evaluate(opened(), s(32001)).decision.unknown, ["load"]); // later than the sample
});
test("malformed signal values count as missing", () => {
for (const patch of [{ load: { load1: NaN } }, { load: { cpus: 0 } }, { load: { load1: -1 } }, { memory: { limit: 0 } },
{ memory: { available: "1" } }, { oom: { kills: Infinity } }, { disk: { free: undefined } }]) {
const d = evaluate(opened(), sample(32000, patch)).decision;
assert.deepEqual(d.reasons, ["health-unknown"], JSON.stringify(patch));
}
});
test("after health-unknown, recovery waits the full 30 s", () => {
const unknown = evaluate(opened(), sample(32000, { load: null })).state;
const r = run(unknown, every2s(34000, 64000));
assert.equal(r.decisions.at(-2).mode, "held");
assert.equal(r.last.mode, "open");
});
test("a host that never had PSI skips the PSI rows", () => {
const r = run(initialState(), every2s(0, 30000, { psi: null }));
assert.equal(r.last.mode, "open");
assert.deepEqual(r.last.unknown, []);
});
test("PSI that disappears after it was present holds with health-unknown", () => {
const d = evaluate(opened(), sample(32000, { psi: null })).decision;
assert.equal(d.mode, "held");
assert.deepEqual(d.reasons, ["health-unknown"]);
assert.deepEqual(d.unknown, ["psi"]);
// Also when it first appears late, then goes.
const late = run(initialState(), [...every2s(0, 30000, { psi: null }), sample(32000), sample(34000, { psi: null })]);
assert.deepEqual(late.last.unknown, ["psi"]);
// A stale PSI signal counts the same.
const stalePsi = { ...sample(32000), psi: { at: 25000, cpuSome: 0, memoryFull: 0, ioFull: 0 } };
assert.deepEqual(evaluate(opened(), stalePsi).decision.unknown, ["psi"]);
});
test("a rise in the OOM kill counter is critical; the first sample is the baseline", () => {
const first = evaluate(initialState(), sample(0, { oom: { kills: 5 } }));
assert.notEqual(first.decision.mode, "critical");
const r = run(opened(), [sample(32000, { oom: { kills: 0 } }), sample(34000, { oom: { kills: 1 } })]);
assert.equal(r.last.mode, "critical");
assert.deepEqual(r.last.reasons, ["critical-oom"]);
});
test("a falling OOM counter resets the baseline with a note, not an event", () => {
const s = run(initialState(), every2s(0, 30000, { oom: { kills: 6 } })).state;
assert.equal(s.mode, "open");
const r = run(s, [sample(32000, { oom: { kills: 2 } }), sample(34000, { oom: { kills: 2 } })]);
assert.deepEqual(r.decisions[0].notes, ["oom-counter-reset"]);
assert.ok(r.decisions.every((d) => d.mode === "open"));
});
test("memory under 5% and disk under 1 GiB are critical, at their edges", () => {
const five = R.critical.memoryFraction * LIMIT;
assert.equal(evaluate(opened(), sample(32000, { memory: { available: Math.ceil(five) } })).decision.mode, "held");
const mem = evaluate(opened(), sample(32000, { memory: { available: Math.ceil(five) - 1 } })).decision;
assert.equal(mem.mode, "critical");
assert.deepEqual(mem.reasons, ["critical-memory"]);
// On 20 GiB the 5% line is exactly 1 GiB, so the edge itself is tested.
const s20 = (available) => sample(32000, { memory: { available, limit: 20 * GiB } });
assert.equal(evaluate(opened(), s20(1 * GiB)).decision.mode, "held");
assert.deepEqual(evaluate(opened(), s20(1 * GiB - 1)).decision.reasons, ["critical-memory"]);
assert.equal(evaluate(opened(), sample(32000, { disk: { free: 1 * GiB } })).decision.mode, "open");
assert.deepEqual(evaluate(opened(), sample(32000, { disk: { free: 1 * GiB - 1 } })).decision.reasons, ["critical-disk"]);
});
test("critical stays until reconciliation, then still waits the 30 s release", () => {
const crit = evaluate(opened(), sample(32000, { disk: { free: 0 } })).state;
const waiting = run(crit, every2s(34000, 120000));
assert.ok(waiting.decisions.every((d) => d.mode === "critical" && d.reasons[0] === "critical-disk"));
// Reconciled while the condition is still there: still critical.
assert.equal(evaluate(crit, sample(34000, { disk: { free: 0 } }), R, { reconciled: true }).decision.mode, "critical");
// Reconciled and healthy: held, then open 30 s later.
const after = run(waiting.state, every2s(122000, 152000), R, (s) => ({ reconciled: s.at === 122000 }));
assert.deepEqual(after.decisions[0].reasons, ["release-pending"]);
assert.deepEqual(after.decisions[0].critical, []);
assert.equal(after.decisions.at(-2).mode, "held");
assert.equal(after.last.mode, "open");
});
test("critical reasons accumulate while critical", () => {
const r = run(opened(), [sample(32000, { disk: { free: 0 } }), sample(34000, { oom: { kills: 3 } })]);
assert.deepEqual(r.last.reasons, ["critical-disk", "critical-oom"]);
});
test("a clock that goes backwards holds with health-unknown", () => {
const d = evaluate(opened(), sample(20000)).decision;
assert.equal(d.mode, "held");
assert.deepEqual(d.unknown, ["clock"]);
assert.deepEqual(d.notes, ["clock-backwards"]);
});
test("a sample without a numeric time throws", () => {
for (const bad of [null, {}, { at: "1" }, { at: NaN }]) {
assert.throws(() => evaluate(initialState(), bad), (e) => e instanceof SupervisorError && e.kind === "sample");
}
});
test("evaluate doesn't change the state it was given", () => {
const s = Object.freeze({ ...opened(), critical: Object.freeze([]) });
assert.doesNotThrow(() => evaluate(s, sample(32000, { disk: { free: 0 } })));
assert.equal(s.mode, "open");
});
test("resolveRules with no overrides is the D4 table, frozen", () => {
assert.deepEqual(resolveRules(), DEFAULT_RULES);
assert.ok(Object.isFrozen(DEFAULT_RULES) && Object.isFrozen(DEFAULT_RULES.psi.hold));
assert.deepEqual(DEFAULT_RULES.load, { hold: 0.8, release: 0.55, samples: 3 });
assert.equal(DEFAULT_RULES.releaseMs, 30000);
assert.equal(DEFAULT_RULES.staleMs, 6000);
});
test("overrides apply in the evaluator", () => {
const rules = resolveRules({ releaseMs: 10000, load: { samples: 1 } });
const r = run(initialState(), every2s(0, 10000), rules);
assert.equal(r.last.mode, "open");
assert.equal(evaluate(r.state, sample(12000, { load: { load1: 0.9 } }), rules).decision.mode, "held");
});
test("contradictory or unknown settings refuse", () => {
const cases = [
[{ load: { release: 0.8 } }, /load.release must be above 0 and below load.hold/],
[{ load: { hold: 0.5 } }, /load.release must be above 0 and below load.hold/],
[{ memory: { releaseFraction: 0.15 } }, /releaseFraction must be above/],
[{ memory: { releaseBytes: 1 * GiB } }, /releaseBytes must be above/],
[{ memory: { holdFraction: 1.2 } }, /between 0 and 1/],
[{ psi: { release: { ioFull: 10 } } }, /psi.release.ioFull must be above 0 and below psi.hold.ioFull/],
[{ psi: { hold: { cpuSome: 120 } } }, /percentages/],
[{ psi: { samples: 0 } }, /psi.samples must be an integer/],
[{ load: { samples: 1.5 } }, /load.samples must be an integer/],
[{ staleMs: 2000 }, /staleMs must be above sampleMs/],
[{ releaseMs: -1 }, /releaseMs must not be negative/],
[{ critical: { memoryFraction: 0.15 } }, /critical.memoryFraction/],
[{ critical: { diskBytes: 0 } }, /critical.diskBytes/],
[{ loadd: {} }, /unknown key loadd/],
[{ psi: { hold: { gpu: 1 } } }, /unknown key psi.hold.gpu/],
[{ load: { toString: 1 } }, /unknown key load.toString/],
[{ constructor: {} }, /unknown key constructor/],
[JSON.parse('{"memory":{"__proto__":{}}}'), /unknown key memory.__proto__/],
[{ load: { hold: "0.8" } }, /load.hold must be a finite number/],
[{ load: 1 }, /load must be an object/],
[[], /rules must be an object/],
];
for (const [over, re] of cases) {
assert.throws(() => resolveRules(over), (e) => e instanceof SupervisorError && e.kind === "rules" && re.test(e.message), JSON.stringify(over));
}
});