Candidate morgana/row24494ab4c2feona36523713, manifest 6e7e3e3c (8 paths). Webby APPROVE r1, rev 3008 (webby-0084, REVIEW-r1.md at webby/reviews ebb3d83bf; request #1757 c30154). Gate on Sage's slot at81e037d20(sage-2147): the five gate-lane files whole 51/0, test-task.sh 91/0, test-release.sh 14/0, mutants M-G, M-L, M-A, M-T and M-F killed.94ab4c2feadds only BUILD.md results. Webby's drift check: a36523713..d37210288 changes none of the 8 paths and no path either suite reads; merge-tree clean. Rulings A and C (sage-1914, sage-2061). The dropped real-model coverage goes to DEFERRED in the follow-up doc commit.
16 KiB
Row 244: gate runs use a fixture pi auth file
Brief: docs/plans/2026-10-21_gate-env-rows.md § "Gate runs use a fixture pi auth file" (Sage, sage-1909).
Reviewer Webby. DEFERRED entry "Gate runs mount the host's pi auth file" (059577886). Design notes
written before row 197 lands, read against origin/herb/row197-r6 (df0c59608). Recheck every line cite
against 197's landing before code starts.
Where Compose is reached
compose.yaml:43 mounts ${PI_AUTH_FILE:-/home/jwoltje/.pi/agent/auth.json} read-only. The default is
a literal path, not $HOME, so keeping HOME in the lane env doesn't change it.
Gate steps come from discoverSteps (gate-lane.mjs:32): one pkg-<name> step per package with tests
(not queue) and one script-<name> step per scripts/test-*.sh. A grep of those for run-task.sh,
docker compose, hello.sh, verify.sh, agent.sh and release.sh activate finds two Compose
paths. The other hits (cohort.test.mjs "hello" shim requests, test-auth.sh) don't reach Compose.
| Step | Path to Compose | Adapter |
|---|---|---|
test-task.sh :150-385 |
scripts/run-task.sh run → mosaic-task.mjs:396 docker compose run with spawnEnv = {...process.env} |
mock (mock_config) |
test-release.sh :69, :85, :93 |
release.sh activate → health_check (:41-55) → run-task.sh run tasks/hello-marker.json |
pi (config :61 sets no adapter; provider zai, model glm-5.3-flash) |
mosaic-task.mjs passes process.env through, so a PI_AUTH_FILE exported by a suite reaches Compose.
The conflict: test-release's healthy activation is a live provider call
"healthy activation succeeds" (:85) and "repeat activation succeeds" (:93) need health_check to
pass. That runs the real pi adapter, asks glm-5.3-flash for MOSAIC_HELLO_OK, and authenticates through
the mounted auth file. The lane's cleanEnv drops ZAI_API_KEY, so the mount is the only credential.
With a fixture auth file the provider call fails, and four checks fail: :85, the pointer check after it,
:93 and "log is append-only". So every gate run today spends a real provider call on the host credential.
Options put to Sage (morgana-0037):
- A (recommended). test-release's config sets
"adapter":"mock", and the two healthy activations run withMOSAIC_MOCK_RESPONSE=MOSAIC_HELLO_OK. The fault injection still refuses, because its task expectsMOSAIC_FAULT_<random>. The suite keeps proving the release state machine, the image and Compose path, and the pointer and log. It stops proving that the pi image can reach the provider. Realrelease.sh activateruns still do that, with the real credential. No credential and no spend in the gate. - B. Keep the live call. The lane passes the real auth file explicitly. That contradicts the brief and DEFERRED, so I don't take it without a ruling.
Ruling: A (sage-1914, 09:51Z). B keeps a live billed call on every gate, which is what this row stops. Sage's conditions:
- test-release.sh's suite config sets
"adapter":"mock", with a comment beside it saying why: no credential and no provider call in the gate (row 244). The healthy activations run withMOSAIC_MOCK_RESPONSE=MOSAIC_HELLO_OK. - Fault injection still refuses. It keeps the random
MOSAIC_FAULT_<n>expectation, and the mock's fixed reply can't satisfy it. Mutant M-F forces the fault task's expected value toMOSAIC_HELLO_OK. It must fail the suite. - The fake-docker test also asserts that test-release's activations ran with the mock adapter. The
fake docker records
MOSAIC_ADAPTER, whichmosaic-task.mjssets inspawnEnvfrom the resolved config. Then a suite drifting back to pi fails the test. - Coverage dropped: test-release.sh no longer proves that the pi image reaches the provider. A real
release.sh activatestill proves it, with the real credential. Sage adds a DEFERRED line at landing if Webby wants one.
guardDocker's new refusal also gets a mutant (M-G, below).
Change
scripts/test-task.shandscripts/test-release.sh: right afterSANDBOXis made, write$SANDBOX/pi/auth.json({}, mode set bychmod 600after the write, so it doesn't depend on the umask, row 243) andexport PI_AUTH_FILE. It has to be a regular file that exists, because a missing bind source makes Docker create a directory on the host. The suites set it themselves, so the legacy gate (outside the lane) is covered too.scripts/test-release.sh: option A above.scripts/testing/gate-lane.mjscleanEnv: setPI_AUTH_FILEtojoin(o.tempRoot,'pi-auth.json'), andrunwrites that file ({},flag:'wx', then chmod 0600) next to where it writesbin/docker. Any later step that reaches Compose without its own fixture then gets the lane's fixture.scripts/testing/lane-docker.mjsguardDocker:compose runalso refuses unlessPI_AUTH_FILEis set, resolves insiderun.tempRootand isn't the host default (refusal textCompose auth mount not a lane fixture). The guard already sees every Docker call a lane step makes, so this is where "no gate step mounts the host file" is enforced at run time. The CLI entry passesprocess.env.compose.yamlis unchanged (brief: not in scope).- At landing: the DEFERRED entry moves to Done with the landing commit.
Tests
In packages/cli/tests/gate-lane.test.mjs (row 197's file; the exact file follows 197's landing):
guardDockerrefusescompose runwithPI_AUTH_FILEunset, set to the host default, or set outsidetempRoot; it accepts the lane fixture and a suite fixture undertempRoot. Other verbs are unchanged.cleanEnvreturns aPI_AUTH_FILEinsidetempRootthat isn't the host default.- The suites, read through their Compose environment: run
test-task.shandtest-release.shwith a fakedockerfirst onPATH. It answersinfoandimage inspectwith 0, and forcompose runit appendsPI_AUTH_FILEand whether that file is a regular 0600 file to a record. Then it prints the mock response. The test ignores the suites' own exit code and asserts that the record has at least onecompose runfrom each suite and that every entry names an existing fixture file, never/home/jwoltje/.pi/agent/auth.json. The test never reads the host file. If a full suite run with the fake docker turns out too slow or brittle, I'll say so in the review request, with timings, and fall back to the guard andcleanEnvtests plus a static check that each suite exportsPI_AUTH_FILEbefore its first Compose call. I won't drop it silently.
Mutants (each must fail a test; run on Sage's slot):
- M-F: the fault task's
expectExactforced toMOSAIC_HELLO_OKinrelease.shhealth_check. Must failtest-release.sh("fault-injected activation refuses"). - M-G: the
guardDockerPI_AUTH_FILEcheck removed. Must fail the guard test. - M-A: test-release's config back to no adapter (pi). Must fail the fake-docker test (condition 3).
- M-L:
cleanEnvstops settingPI_AUTH_FILE. Must fail thecleanEnvtest.
Gate: those tests, test-task.sh and test-release.sh whole, the 197 lane tests, and the four
mutants, all on Sage's slot.
Files (expected)
scripts/test-task.sh, scripts/test-release.sh, scripts/testing/gate-lane.mjs,
scripts/testing/lane-docker.mjs, packages/cli/tests/gate-lane.test.mjs (or a new
gate-lane-auth.test.mjs), docs/plans/DEFERRED.md (at landing), scripts/testing/README.md if it
lists the guard's rules.
Correction (morgana-0040, 18:50Z)
The table above says test-task.sh uses only the mock adapter. That was wrong. On a36523713 two
sections run the pi adapter through $CONFIG (provider zai) and authenticate through the host mount:
:443-501 (live hello, result.json written and correct, wrong expectExact, distinct run dirs, list,
fork base, fork child, fork recall, ancestor untouched, child branch file) and :518-530 (user recall
run, recalled the user name). That's 13 checks and about 6 billed provider calls per gate run. I missed
them because I read the mock section and stopped there.
Ruling C (sage-2061, 18:50Z). D (keep them live) refused.
- The run-record checks (hello, result.json, wrong expectExact, distinct run dirs, list) move to the mock adapter. Every assertion is kept. Only the response source changes.
- Fork recall and user recall run only under
MOSAIC_LIVE_PROVIDER=1. Otherwise the suite printsskip live provider cases (row 244: no provider call in the gate), which the lane's requiredSkips regex doesn't match, on purpose. A test asserts the line prints when the variable is unset and that the live checks run, and fail under a fixture, when it's 1 (M-T). - The dropped coverage and the DEFERRED wording are below. Sage writes the entry at landing.
Build (on a36523713, sage-2056)
Rechecked against 197's landing: lane-docker.mjs:30 guardDocker, gate-lane.mjs:109 cleanEnv and
:181 (bin/docker written), compose.yaml:43, release.sh:41-55 health_check, mosaic-task.mjs:321-322
spawnEnv and MOSAIC_ADAPTER, test-release.sh:61 config. All hold.
scripts/testing/lane-docker.mjs:guardDocker(argv,run,env=process.env). After the option check, acompose runcallslaneAuthFile. It refuses (Compose auth mount not a lane fixture) unlessPI_AUTH_FILEis absolute, isn't the host default, resolves (realpath) inside the realpath ofrun.tempRoot, and is a regular file. Symlinks out of the root and relative paths refuse.infoandimage inspectneed no auth file. The CLI entry passesprocess.env.scripts/testing/gate-lane.mjs:cleanEnvsetsPI_AUTH_FILE=join(o.tempRoot,'pi-auth.json').runGatewrites it ({}, flagwx, thenchmodSync0600) right afterbin/docker.MOSAIC_LIVE_PROVIDERisn't on cleanEnv's keep list, so the lane can't request live cases.scripts/test-release.sh: exports a$SANDBOX/pi/auth.jsonfixture ({},chmod 600after the write, exits 1 if the write fails). The config adds"adapter":"mock"with the ruling A comment. The fault, healthy and repeat activations all run withMOSAIC_MOCK_RESPONSE=MOSAIC_HELLO_OK, so the fault activation refuses only because itsMOSAIC_FAULT_<n>expectation is wrong. (Correction, before any run: the first build gave the fault activation no mock response. The mock then answered empty, which fails any expectation, so M-F would have survived. Fixed in the follow-up commit.)scripts/test-task.sh: same fixture, exported only whenMOSAIC_LIVE_PROVIDERisn't 1, so an operator's live run keeps their ownPI_AUTH_FILE(or Compose's default). Ruling C (a) and (b) as above. The section header changes from "live: real runs" to "run records through the mock adapter", the check name "live hello task..." becomes "hello task...", and the docker-less branch of the new split printsskip run-record cases (docker unavailable).packages/cli/tests/gate-lane-image.test.mjs(:21, :106, :119): the compose-run calls passrun.env, the composeFixture env, which already holds a 0600PI_AUTH_FILEinside tempRoot (accepted by sage-2061).scripts/testing/README.md: one sentence on the lane fixture and the refusal.- New
packages/cli/tests/gate-lane-auth.test.mjs, five tests:- guardDocker accepts the lane fixture and a suite fixture under tempRoot. It refuses unset, empty,
the host default, a file outside, a symlink inside that points outside, a missing path, a directory
and a relative path. The option check still comes first.
infoandimage inspectpass with no env. - cleanEnv sets the lane fixture path and drops
MOSAIC_LIVE_PROVIDERand the caller'sPI_AUTH_FILE. - test-release.sh in a private copy of the checkout, with a fake
dockerfirst on PATH. The fake answers every verb with 0. Forcompose runit recordsPI_AUTH_FILE,stat -c '%F %a'of it (never for the host path) andMOSAIC_ADAPTER, and it printsMOSAIC_MOCK_RESPONSEwhen the adapter is mock. Asserts: at least 3 compose runs, every one on a regular 0600 file under the suite's TMPDIR, and every adapter mock (condition 3). The healthy and fault lines print OK. - test-task.sh, same harness: every compose run on its fixture and the mock adapter, the skip line printed exactly twice, and the hello check OK.
- M-T: test-task.sh with
MOSAIC_LIVE_PROVIDER=1and an operator-style fixture. No skip line,FAIL fork base: teach succeedsandFAIL recalled user name, at least one pi run, and every run on the operator's file. The copy leaves out.git,node_modules(symlinked),v1,agents,docs,BUILD-LOG.mdandscripts/foundation, about 14 MB. Copying is needed becausetest-release.shrewritesRELEASEwhile it runs, which would race other cli tests that read it. The suites ignore their own exit codes here: the fake doesn't emit the mock adapter's env lines, so some seam checks fail by design.
- guardDocker accepts the lane fixture and a suite fixture under tempRoot. It refuses unset, empty,
the host default, a file outside, a symlink inside that points outside, a missing path, a directory
and a relative path. The option check still comes first.
- Readers:
gate-lane.mjsis imported by gate-lane.test, gate-lane-state.test, gate-lane-fixtures.test, gate-lane-image.test and fixtures/gate-lane-child.mjs (unchanged exports).lane-docker.mjsby gate-lane-image.test. No test hash-pins any changed file. vars.test.mjs namesscripts/test-task.shonly as a string value.
Coverage dropped from the gate (for Webby, and for the DEFERRED entry):
test-release.shno longer proves that the pi image reaches the provider (ruling A).test-task.shno longer proves session fork recall or onboarding user recall against a real model (ruling C).
DEFERRED wording for Sage (sage-2061 condition 3): "Real-model suite cases run outside the gate.
Row 244 moved test-task.sh fork recall (M11) and user recall (M16) behind MOSAIC_LIVE_PROVIDER=1,
and test-release.sh activations to the mock adapter. Before any re-pin whose tree changes
packages/harness or the session fork code (scripts/mosaic-task.mjs session and sessionForkFrom
handling, the pi adapter), an operator runs MOSAIC_LIVE_PROVIDER=1 bash scripts/test-task.sh with the
real pi credential and a real release.sh activate, and records both results on the re-pin row."
Mutants (each must fail a named test; on Sage's slot):
- M-F:
release.shhealth_check fault taskexpectExactforced toMOSAIC_HELLO_OK. Fails test-release.sh "fault-injected activation refuses", and test 3's OK match. This needs the fault activation'sMOSAIC_MOCK_RESPONSE=MOSAIC_HELLO_OK(see the correction above). - M-G: the
laneAuthFile(env,run)call removed from guardDocker. Fails test 1. - M-A:
,"adapter":"mock"removed from test-release's config. Fails test 3 (adapter set is pi). - M-L: the
PI_AUTH_FILEkey removed from cleanEnv. Fails test 2. - M-T: test 5 itself (sage-2061). As a mutant check too: the
MOSAIC_LIVE_PROVIDERgate in test-task forced on fails test 4 (skip line count and pi adapter).
Results, Sage's slot (sage-2147), candidate 81e037d20, 2026-10-21 20:46:16Z to 20:46:59Z. Busy check
CLEAR before and after. Image mosaic-poc-agent:0.85.1-r0.0.12 present, so nothing was built. No
unexpected failure. Singles ran under umask 022 with TMPDIR under /mnt/storage/scratch/tmp.
| Run | rc | Result | Time |
|---|---|---|---|
| gate-lane-auth.test.mjs (new) | 0 | 5 pass, 0 fail | 14 s |
| gate-lane-image.test.mjs | 0 | 11 pass, 0 fail | 0 s |
| gate-lane.test.mjs | 0 | 13 pass, 0 fail | 0 s |
| gate-lane-state.test.mjs | 0 | 14 pass, 0 fail | 2 s |
| gate-lane-fixtures.test.mjs | 0 | 8 pass, 0 fail | 0 s |
| test-task.sh | 0 | OK 91, FAIL 0, the two live-skip lines | 12 s |
| test-release.sh | 0 | OK 14, FAIL 0, Docker section on the mock adapter | 2 s |
Mutants, each run on its named gate-lane-auth test and restored after:
| Mutant | Target | Result |
|---|---|---|
| M-G | test 1 | killed (0 pass, 1 fail) |
| M-L | test 2 | killed (0 pass, 1 fail) |
| M-A | test 3 | killed (0 pass, 1 fail) |
| M-T | test 4 | killed (0 pass, 1 fail), 6 s |
| M-F | test 3 | killed (0 pass, 1 fail) |
| M-F | real test-release.sh | killed: rc 1, OK 11, FAIL 3, the first fault-injected activation refuses (exit 0, expected 1) |