81 lines
4.8 KiB
Markdown
81 lines
4.8 KiB
Markdown
# Supervisor
|
|
|
|
Q198 S1 (row 257, #1762): parsers for the host pressure inputs in design D4
|
|
(`docs/plans/2026-10-21_q198-system-manager.md`), and a pure evaluator that turns timestamped samples
|
|
into a hold or release decision. Nothing here samples the host, runs a process or timer, or enforces
|
|
anything. The live sampler, the Console view and admission are S2 and S3.
|
|
|
|
```sh
|
|
node --test packages/supervisor/tests/
|
|
```
|
|
|
|
## Parsers
|
|
|
|
Each parser takes file text the caller already read. Text that isn't the shape the kernel writes throws
|
|
a `SupervisorError` with `kind: "parse"`; a sampler treats that as a missing signal.
|
|
|
|
| Export | Input | Returns |
|
|
|---|---|---|
|
|
| `parseLoadavg` | `/proc/loadavg` | `{ load1, load5, load15 }` |
|
|
| `parseMeminfo` | `/proc/meminfo` | `{ memTotal, memAvailable, swapTotal, swapFree }` in bytes |
|
|
| `parsePsi(text, resource)` | `/proc/pressure/{cpu,memory,io}` or a cgroup `*.pressure` | `{ some, full }`, each `{ avg10, avg60, avg300, total }`; `full` is null only for cpu |
|
|
| `parseCpuList` | `/sys/devices/system/cpu/online`, `cpuset.cpus.effective` | how many CPUs the list names |
|
|
| `parseCpuMax` | `cpu.max` | the quota in CPUs, or null for `max` |
|
|
| `parseMemoryMax` | `memory.max` | bytes, or null for `max` |
|
|
| `parseMemoryCurrent` | `memory.current` | bytes |
|
|
| `parseMemoryEvents` | `memory.events` | `{ oom, oomKill }` |
|
|
| `parseVmstat` | `/proc/vmstat` | `{ pswpin, pswpout, oomKill }`; `oomKill` is the host-only OOM counter, null when absent |
|
|
|
|
`effectiveCpus({ online, cpuset, cpuMax })` is the smallest of the host count, the cpuset and the quota
|
|
at every cgroup level given; a fractional quota stays fractional. `effectiveMemory({ meminfo, levels })`
|
|
takes cgroup levels as `{ max, current }` text. The limit is the smallest of `MemTotal` and every numeric
|
|
`memory.max`. Available memory is the smallest of `MemAvailable` and `max - current` at each limited level.
|
|
`memory.current` counts page cache, so this errs toward holding. With no cgroup input both functions use
|
|
the host's figures.
|
|
|
|
## Evaluator
|
|
|
|
`evaluate(state, sample, rules, { reconciled })` returns `{ state, decision }`. It starts from
|
|
`initialState()`, which is held: no signal has been seen yet. Time is the sample's `at` in milliseconds.
|
|
The evaluator never reads the clock.
|
|
|
|
A sample has these signals: `load { at, load1, cpus }`, `memory { at, available, limit }`,
|
|
`psi { at, cpuSome, memoryFull, ioFull }` (avg10 percentages), `oom { at, kills }` and `disk { at, free }`.
|
|
|
|
| Signal | Hold | Release |
|
|
|---|---|---|
|
|
| `load1 / cpus` | ≥ 0.80 on 3 consecutive samples | ≤ 0.55 |
|
|
| available memory | < max(15% of limit, 1 GiB), on one sample | > max(25% of limit, 2 GiB) |
|
|
| PSI avg10 | cpu some ≥ 20, memory full ≥ 2 or io full ≥ 10, on 3 consecutive samples | cpu some < 10, memory full < 0.5 and io full < 3 |
|
|
| critical | OOM kill counter rises, memory < 5% of limit, or disk free < 1 GiB | the caller passes `reconciled`, then the normal release |
|
|
|
|
- **Release** needs every applicable release condition on every sample for 30 s. A dead-band
|
|
sample (between hold and release) or a new hold restarts the window.
|
|
- **Missing or stale:** a required signal that is missing, malformed, more than 6 s older than the
|
|
sample, or later than it holds at once with `health-unknown`. `load`, `memory`, `oom` and `disk` are
|
|
always required. PSI becomes required once a sample has carried it. A host without PSI skips the PSI
|
|
rows, and one that loses it holds.
|
|
- **Consecutive counts** reset after a gap of more than 6 s between samples. The PSI count runs across
|
|
resources: any PSI hold condition on 3 consecutive samples holds.
|
|
- **OOM:** the first OOM sample is the baseline. A counter that falls (a recreated cgroup) resets the
|
|
baseline with the note `oom-counter-reset`.
|
|
- **Critical** never ends on thresholds alone. `reconciled` only counts once no critical condition
|
|
remains, and the 30 s release still follows. The evaluator never sets it and never touches a kill
|
|
switch.
|
|
- **A clock that goes backwards** holds with `health-unknown` and the note `clock-backwards`.
|
|
|
|
`decision` is `{ mode, reasons, unknown, critical, releaseAt, notes }`.
|
|
- `mode` is `open`, `held` or `critical`.
|
|
- `reasons` are `health-unknown`, `load-high`, `memory-low`, `psi-cpu`, `psi-memory`, `psi-io`,
|
|
`hysteresis`, `release-pending` or the critical codes `critical-oom`, `critical-memory` and
|
|
`critical-disk`.
|
|
- `releaseAt` is set while a release is pending.
|
|
|
|
`DEFAULT_RULES` is the D4 table, frozen. `resolveRules(overrides)` merges overrides key by key. It refuses
|
|
with `kind: "rules"` on an unknown key, a non-number or a contradictory pair, for example:
|
|
- a release threshold at or past its hold;
|
|
- a release floor at or below its hold floor;
|
|
- a critical memory fraction at or above the hold fraction;
|
|
- a stale limit at or below the sample interval.
|
|
Provider scope (429, quota, Retry-After) is not here; it belongs to S3 and S5.
|