mosaicstack/stack
25
0 Fork 1
Code Issues 403 Pull Requests 13 Packages Projects Releases 15 Wiki Activity

row 292 (#1797) packet: targeted mutant kill rules per sage-3152 (pattern, stack lines and text per mutant; anything else is a STOP; a survivor gets one whole-file run), short TMPDIR per stage, run plan after Duckworth's round

Browse Source
This commit is contained in:
jason.woltje committed 2026-10-22 21:50:51 -05:00
1 parent f8b9c2f73e
commit 3fd8f42f64
2 files changed
+64 -36
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

agents/gizmoduck/work/queue-292/BUILD.md
+49 -24
View File
@@ -164,35 +164,60 @@ the round key.
## Mutants
agents/gizmoduck/work/queue-292/mutants.py. Each anchor appears exactly once, and each mutated
script passes `bash -n` (checked with python3 and bash only, no node). The kill lines below come
from reading the tests. None has run.
script passes `bash -n` (checked with python3 and bash only, no node). None has run. M1-M5 are
Gander's edits (changes.json) on the resolved function. M6-M9 test the resolution. M1 restores the
console-only skip, the brief's required mutant.
| id | edit | should die in |
|---|---|---|
| M1 | restore the console-only skip (brief's required mutant) | config-any-key non-console refusal; the pre-270 test; row 203 federation |
| M2 | ignore a failed target validate | config-any-key non-console refusal (exit 0, not 3) |
| M3 | no-loader target passes any later key | config-any-key missing-loader |
| M4 | malformed JSON passes as absent | config-any-key malformed JSON |
| M5 | skip the loader on an oldest-key config | config-any-key accepting loaders (no witness line) |
| M6 | probe drops the federation preference | missing-loader federation case (gets the later-key text) |
| M7 | probe drops the console preference | missing-loader console case |
| M8 | final refusal treats only console as 203's | row 203 federation test; missing-loader federation case |
| M9 | no-loader branch treats only console as 203's | missing-loader federation case |
### Kill rules (sage-3152)
M1-M5 are Gander's edits (changes.json) on the resolved function. M6-M9 test the resolution.
Each mutant runs only its killing test: `node --test --test-name-pattern '<pattern>'
tests/stack-release.test.mjs`. The patterns are the test names, anchored with ^ and $, and all
four are top-level tests. A mutant counts as killed only when all three of these hold:
## Run plan (on Sage's CLEAR only, C2)
1. The run ends with `ℹ fail 1`, and the failing test is the named one. The spec reporter lists
it twice, inline and in the failing-tests summary.
2. The first error's stack names every listed stack-release.test.mjs line.
3. The log holds the listed text. The texts carry no quote characters, which node's inspect may
escape.
Gate per the brief. stack-release.test.mjs whole, the cli suite whole and the ten
`scripts/test-*.sh`, then M1-M9, each on stack-release.test.mjs whole. Every mutant gets
`git checkout`, sha256 equal to the candidate and a clean tree before the next. TMPDIR from
`mktemp -d /mnt/storage/scratch/tmp/gXXX`, pipefail, tee, `timeout -k 10` everywhere.
Anything else is a STOP that goes to Sage before the next mutant: a fail in another test or at
another line, a missing text, zero tests matched, a timeout or a crash before the assertion. A
fixture or module-load error is the second-fence trap from 264a M2. A mutant whose targeted run
passes gets one run of stack-release.test.mjs whole before it counts as a survivor.
Limits from the run 5 receipts (gizmo-271-out/r4, 01:49Z 10-23): stack-release.test.mjs whole took
682 s (duration_ms 681648), the cli suite took 624 s, and the longest shell suite took 44 s. At 3x:
2100 s for stack-release.test.mjs, alone or as a mutant run, 2100 s for the cli suite and 900 s per
shell suite. Expect about 12 + 11 + 3 minutes for the base and about 9 x 12 minutes for the mutants,
so roughly 2.2 hours in all. Sage may prefer a shorter mutant run; this packet doesn't assume one.
| id | edit | test (pattern) | stack lines | text | why that assertion |
|---|---|---|---|---|---|
| M1 | console-only skip restored | non-console target refusal | :517 (refused), :2619 | `actual: 0` | no console key, so the check returns 0 and activate B runs: exit 0, not 3 |
| M2 | failed validate ignored (`\|\| true`) | non-console target refusal | :517, :2619 | `actual: 0` | the stand-in's exit 2 is lost, so activate B runs |
| M3 | no-loader target passes any later key | missing-loader copies | :2670 | `actual: 0` | compactionReadBytes, the first case, passes with status 0 |
| M4 | malformed JSON treated as absent | missing config passes but malformed JSON | :2684 | `actual: 0` | key 2 falls through to the witness loader, which exits 0 |
| M5 | oldest-key config skips the loader | accepting loaders run validate | :2654 | `actual: []` | the witness file is never written |
| M6 | probe drops the federation preference | missing-loader copies | :2677 | `to validate configuration key: ` | federation after compactionStreams: the probe picks compactionStreams |
| M7 | probe drops the console preference | missing-loader copies | :2677 | `to validate configuration key: ` | the same for console, the first case of that loop |
| M8 | final refusal treats only console as 203's | missing-loader copies | :2677 | `is refused by old` | federation gets Gander's text, not KEY_MSG |
| M9 | no-loader branch treats only console as 203's | missing-loader copies | :2677 | `to validate configuration key: ` | federation's reason gains the later-key wording |
M1 would also fail the pre-270 test and the row 203 federation test, and M8 the row 203 federation
test. The targeted runs don't select those, so they aren't counted.
## Run plan (on Sage's CLEAR only, after Duckworth approves this round)
1. stack-release.test.mjs whole, limit 2100 s.
2. The cli suite whole, `node --test tests/*.test.mjs` in packages/cli, limit 2100 s.
3. The ten `scripts/test-*.sh`, limit 900 s each.
4. Only on a green base: M1-M9 under the kill rules above. Targeted runs get 600 s and a
whole-file rerun of a survivor gets 2100 s. After each mutant: `git checkout --
scripts/stack-release.sh`, sha256 equal to the candidate's (8f535d53...) and a clean tree.
Every stage gets its own short TMPDIR, `mktemp -d /mnt/storage/scratch/tmp/gXXX`, directly under
/mnt/storage/scratch/tmp (sage-3152, 574 run 1's socket-path fault). Every mutant run gets one too.
All runs use pipefail, tee and `timeout -k 10`. The busy check passes before each start. A first red
in stages 1-3 stops the run.
The limits come from the run 5 receipts (gizmo-271-out/r4, 01:49Z 10-23). stack-release.test.mjs
whole took 682 s (duration_ms 681648), the cli suite 624 s and the longest shell suite 44 s. Each
limit is at least 3x. Expected time: about 26 minutes for the base, plus about a minute per targeted
mutant (fixture setup and one test, not measured yet), plus 12 minutes per whole-file rerun.
C2 (duckworth-0229, sage-3146 item 4): every B, C1 and C release now validates its copied real
loader. Any red in an old cli test is a suspect from this change. It stops the run and goes to Sage
agents/gizmoduck/work/queue-292/mutants.py
+15 -12
View File
@@ -3,8 +3,7 @@
# the same edits on the resolved config_checks; M1 restores the console-only skip (the brief's
# required mutant). M6-M9 are the resolution's: row 203's console and federation text. `apply <id>`
# edits the worktree and prints the touched path; each edit asserts its anchor appears exactly once.
# `check` prints each anchor's count and kill. K names the test that should fail (checked by reading,
# not run). A restore is `git checkout -- <path>`, then sha256 against the blob in BUILD.md.
# `check` prints each anchor's count and kill. K is read from the tests, not run. A restore is `git checkout -- <path>`, then sha256 against the blob in BUILD.md.
import sys
WT = "/mnt/storage/scratch/gizmo-292/"
S = "scripts/stack-release.sh"
@@ -25,23 +24,27 @@ M = {
"M9": (S, '[ "$extra" = \'"federation"\' ]; then out="it has no scripts/mosaic-config.mjs"',
'[ "$extra" = \'"federation-off"\' ]; then out="it has no scripts/mosaic-config.mjs"'),
}
# K: (--test-name-pattern, the test.mjs lines the failing stack must show, the text the failure
# must show). Kill rule (sage-3152): the named test fails, its first error's stack names every
# listed line, and the log holds the text (no quote characters, which node's inspect may escape). A fail anywhere else, or a crash first, is a STOP to Sage.
# A survivor of its targeted run gets one whole-file run before it counts as a survivor.
K = {
"M1": "config-any-key: a non-console target refusal stops activate and rollback before any stop; also the pre-270 test and row 203's federation test",
"M2": "config-any-key: a non-console target refusal stops activate and rollback before any stop (exit 0, not 3)",
"M3": "config-any-key: missing-loader copies accept only the four oldest keys (compactionReadBytes passes)",
"M4": "config-any-key: missing config passes but malformed JSON refuses without calling the loader",
"M5": "config-any-key: accepting loaders run validate with the selected config (no witness line on an oldest-key config)",
"M6": "config-any-key: missing-loader copies ... (federation after compactionStreams gets the later-key text)",
"M7": "config-any-key: missing-loader copies ... (console after compactionStreams gets the later-key text)",
"M8": "row 203: a config whose federation key the target loader refuses ... (Gander's text, not KEY_MSG); also the missing-loader federation case",
"M9": "config-any-key: missing-loader copies ... (federation gets 'to validate configuration key')",
"M1": ('^config-any-key: a non-console target refusal stops activate and rollback before any stop$', [517, 2619], "actual: 0"),
"M2": ('^config-any-key: a non-console target refusal stops activate and rollback before any stop$', [517, 2619], "actual: 0"),
"M3": ('^config-any-key: missing-loader copies accept only the four oldest keys$', [2670], "actual: 0"),
"M4": ('^config-any-key: missing config passes but malformed JSON refuses without calling the loader$', [2684], "actual: 0"),
"M5": ('^config-any-key: accepting loaders run validate with the selected config on activate and rollback$', [2654], "actual: []"),
"M6": ('^config-any-key: missing-loader copies accept only the four oldest keys$', [2677], "to validate configuration key: "),
"M7": ('^config-any-key: missing-loader copies accept only the four oldest keys$', [2677], "to validate configuration key: "),
"M8": ('^config-any-key: missing-loader copies accept only the four oldest keys$', [2677], "is refused by old"),
"M9": ('^config-any-key: missing-loader copies accept only the four oldest keys$', [2677], "to validate configuration key: "),
}
cmd = sys.argv[1]
if cmd == "list":
print(" ".join(M)); sys.exit(0)
if cmd == "check":
for k, (f, a, b) in M.items():
print(k, f, open(WT + f).read().count(a), "kill:", K[k])
print(k, f, open(WT + f).read().count(a), "kill:", *K[k])
sys.exit(0)
f, a, b = M[sys.argv[2]]
s = open(WT + f).read()