11 KiB
Row 257: Q198 S1 pressure parser and pure rules evaluator
Issue #1762. Brief: docs/plans/2026-10-21_post-v1-row-briefs.md, "Brief: Q198 S1 pressure parser and
pure rules evaluator" and "Gate: Q198 S1" (cca206cdc). Design: docs/plans/2026-10-21_q198-system-manager.md
r6.1, section 8 row S1 and section 4 (D4). Owner Morgana, reviewer Filbert, gate owner Sage (sage-2196).
Post-v1: MVP rows and re-pin windows go ahead of it on the slot. Base: refactor cca206cdc.
Recorded choices (sage-2196 accepted each)
- Placement. A new package,
packages/supervisor(@mosaic/supervisor), shaped like packages/runs. Nothing on refactor parsed loadavg, PSI, meminfo or cgroup limits (git grep; the only hit was an awk line in scripts/case2-precheck.sh). S2 adds the sampler process to the same package. Packages aren't registered centrally, so no other file changes. - Provider scope (429, quota, Retry-After, the half-open probe) is out. The brief puts it in S3 and S5.
- PSI is not required on a host that has never shown it, so a kernel built without PSI skips the
PSI rows instead of holding forever. Once a sample has carried PSI, it is required: a later missing
or stale PSI signal holds with
health-unknown. That is the case sage-2196 asked for.- Tradeoff: I latch on the first sample that carries PSI, not only on the very first sample. So PSI that appears late and then disappears also holds. This is stricter than "present at the first sample", and it never skips a signal the host has shown.
- Memory under a cgroup. Available memory = min(MemAvailable, limit - memory.current) at each
limited level. memory.current counts page cache, so this errs toward holding.
- Follow-up, not in this row: subtract reclaimable cache using memory.stat
inactive_file, if S2's live samples show holds that the host's MemAvailable doesn't support.
- Follow-up, not in this row: subtract reclaimable cache using memory.stat
Rulings on two readings from the build (sage-2202, re morgana-0057):
- PSI latch. PSI becomes required on the first sample that carries it. This is stricter than "present at the first sample" and fails closed: a PSI file that disappears after it was seen is a missing signal, so health-unknown holds.
- PSI count across resources. Design line 152 says "CPU some ≥20, memory full ≥2, or I/O full ≥10, for 3 samples". The window applies to the whole disjunction, so any PSI condition on 3 consecutive samples holds. If Filbert reads it per resource at review, the review finding decides, and the fix is a single counter.
Other readings of the D4 table, made in the build:
- Memory floors. "Whichever demands more free memory" means the larger floor applies. Hold below max(15% x limit, 1 GiB); release above max(25% x limit, 2 GiB).
- Edges as written. Hold at ≥ for load and PSI, < for memory. Release at ≤ 0.55 for load, < for PSI, > for memory. Critical below 5% of the limit and below 1 GiB of disk.
- Sample counts. "For 3 samples" means 3 consecutive samples. The count resets after a gap of more than 6 s (the stale limit) between samples. The PSI row counts as one row (ruling 6), and the reasons name the resources over on the latest sample.
- Memory holds on one sample. The table's memory row has no sample count.
- Release needs every applicable release condition on every sample for 30 s, by the samples' own
times (
>= 30000ms after the first qualifying sample). A dead-band sample or any hold restarts the window, and so does a gap of more than 6 s. - Start held.
initialState()is held, because nothing has been seen yet. The first healthy sample starts the 30 s window. - Missing or stale. Required signals are load, memory, oom and disk, plus PSI once seen. A signal
holds at once with
health-unknownif it is:- missing;
- non-numeric, negative or zero where a divisor;
- more than 6 s older than its sample (6 s exactly is fresh);
- stamped later than its sample. Disk and OOM are required because the critical row can't be evaluated without them.
- Host-only OOM.
/proc/vmstatoom_killgives the OOM counter when there's no cgroup memory.events. The parser returns null for a kernel without it, and the sampler then has no OOM signal (health-unknown). - Critical latches. It ends only when the caller passes
reconciledwith no critical condition in the sample, then goes through held and the normal 30 s release. The evaluator never setsreconciledand has no kill-switch path. Critical reasons accumulate while latched. - OOM counter. The first OOM sample is the baseline. A falling counter (a recreated cgroup) resets the baseline with a note, not an event.
- Clock going backwards. A sample earlier than the last one holds with
health-unknown(unknownclock, noteclock-backwards). - Swap activity is parsed (
pswpin,pswpout) and has no rule. D4 lists it as a signal with no threshold. - Settings.
resolveRulesrefuses:- unknown keys and non-numbers;
- a release at or past its hold (load, each PSI resource);
- memory release floors at or below the hold floors;
- fractions outside (0, 1);
- PSI values outside 0 to 100;
- sample counts that aren't integers ≥ 1;
- a negative release window;
- a stale limit at or below the sample interval;
- a critical memory fraction at or above the hold fraction;
- a non-positive disk floor.
- Large byte values. Parsed with
Number. A memory.max above 2^53 loses precision, but it only takes part in a min against MemTotal.
Round 2 (Filbert r1 APPROVE c30188 notes, sage-2212)
-
N1:
mergechecked unknown keys withkey in base, so inherited names (toString,constructor,__proto__) passed the check and were dropped instead of refused. It now usesObject.hasOwn(base, key), and the refusal test has those three cases. -
N2: the critical-memory edge was tested only on a 32 GiB limit, where 5% is not an integer, so a
<to<=mutant survived. The test adds a 20 GiB limit, where 5% is exactly 1 GiB, and M8 is that mutant. I checked the other memory edges for the same gap:- The hold floor is non-integer on 32 GiB but is tested at exactly 1 GiB on a 4 GiB limit.
- The release floors are 8 GiB on 32 GiB and 2 GiB on 4 GiB, both integers.
- The disk edge is 1 GiB.
None has the gap.
Files
packages/supervisor/package.json,README.mdsrc/errors.mjs:SupervisorErrorwithkindparse, rules or sample.src/pressure.mjs: the nine parsers,effectiveCpusandeffectiveMemory.src/rules.mjs:DEFAULT_RULES,resolveRules,initialState,evaluate.src/index.mjstests/pressure.test.mjs: parsers on kernel-shaped fixture text, with malformed, truncated and repeated-key cases. Effective CPU cases cover the host-only fallback, cpuset, fractional and nestedcpu.max, andmax. Effective memory cases cover the host-only fallback,max, nested levels, usage over the limit, a limit above MemTotal and a missing memory.current.tests/rules.test.mjs: the D4 table on a fake clock.- Each hold and release threshold on both sides of its edge.
- Three-sample counting and the gap reset.
- The 30 s release at 29 999 and 30 000 ms, and the dead-band restart.
- Missing, stale, future and malformed signals.
- PSI never present, and PSI lost after it was present.
- Critical OOM, memory and disk at their edges, the reconciliation latch and the counter reset.
- A backwards clock, input state left unchanged, and settings refusals and overrides.
tests/purity.test.mjs: a static check that src imports only its own files and uses no fs, child_process, net, os, process, dynamic import, Date or timers.
No test reads /proc, /sys or a cgroup, starts a build, opens a network socket or generates load. No new dependency.
Mutants (each must fail its named test; the brief requires M1 and M2)
| Mutant | Edit | Named test |
|---|---|---|
| M1 hold edge | ratio >= rules.load.hold becomes > |
load holds at 0.80 for 3 samples and not at 0.7999 |
| M2 early release | now - next.releaseSince >= rules.releaseMs becomes >= 0 |
the first state is held, and healthy samples open it after exactly 30 s |
| M3 memory floor | the hold floor Math.max becomes Math.min |
memory holds below the larger floor, on one sample |
| M4 sample count | next.loadCount >= rules.load.samples becomes >= 1 |
load holds at 0.80 for 3 samples and not at 0.7999 |
| M5 stale | the now - signal.at > rules.staleMs check is removed |
a signal is stale at more than 6 s old and fresh at exactly 6 s |
| M6 effective CPU | the quota Math.min becomes Math.max |
effective CPUs take the smallest of host, cpuset and every quota level |
| M7 critical latch | the !reconciled critical branch is removed |
critical stays until reconciliation, then still waits the 30 s release |
| M8 critical memory edge (r2) | available < rules.critical.memoryFraction * limit becomes <= |
memory under 5% and disk under 1 GiB are critical, at their edges |
Gate (Sage's slot, on CLEAR only)
- The three test files, each whole, then
node --test tests/in packages/supervisor as the package suite. - Then M1 to M8, each on its named test, restored after each.
- umask 022, TMPDIR from
mktemp -d /mnt/storage/scratch/tmp/gXXXXXX. - The kit is /mnt/storage/scratch/morgana-r257 (slot257.sh, mutate.py). It refuses unless HEAD is the candidate and the tree is clean, and stops after two unexpected failures (mos-0823).
- Static checks before the slot:
node --checkon every file, run from/. No test has run yet.
Slot result (sage-4159 CLEAR, 2026-10-24 17:00:27Z to 17:00:30Z, candidate 1261fa2d8)
Run as busy.sh && timeout -k 10 1500 slot257.sh in its own session (setsid). The busy check read
CLEAR. The worktree was checked out at 1261fa2d8 with a clean tree. Filbert's approved code is
unchanged. The summary is in gate-run.log and the per-run node output in mutants-run.log.
| Run | tests | pass | fail | skip | cancelled | todo |
|---|---|---|---|---|---|---|
| tests/pressure.test.mjs whole | 15 | 15 | 0 | 0 | 0 | 0 |
| tests/rules.test.mjs whole | 29 | 29 | 0 | 0 | 0 | 0 |
| tests/purity.test.mjs whole | 1 | 1 | 0 | 0 | 0 | 0 |
package suite node --test tests/ |
45 | 45 | 0 | 0 | 0 | 0 |
Mutants: 8 of 8 killed. Each one ran only its named test (1 test, rc=1), and each failure was an AssertionError in that test, not a crash: M1 hold edge, M2 early release, M3 memory floor, M4 sample count, M5 stale, M6 effective CPU, M7 critical latch, M8 critical memory edge.
Restore check: git checkout -- packages/supervisor ran after each mutant, and the script found the
tree clean every time. At the end HEAD was 1261fa2d8 and git status --porcelain was empty. After the
run no node, python or bash process had its cwd under the kit or the worktree, and no g257 TMPDIR was
left behind. The run reached no scripts/test-*.sh.