stack/agents/lena/work/queue-296/BUILD.md

Row 296: FED R7, scratch two-instance acceptance (#1802)

Start

• Started 2026-10-23 18:53:27Z, on Sage's sage-3583 (mos-1489).
• Base: origin/refactor 45dca69f6 (pinned; local copy used, no fetch: fetch runs the credential node script, which is held during Darkwing's re-pin window).
• Branch: quackerjack/row296 in worktree quackerjack-fed-r5-row296.
• Brief: docs/plans/2026-10-23_mvp-readiness-rows.md at 45dca69f6, section "Brief: FED R7, scratch two-instance acceptance" (line 77).
• Constraints from Sage: scratch roots only (mktemp -d under /mnt/storage/scratch/tmp), never ~/.mosaic-dev or the live bus. No push, claim, queue write, node or test run until CLEAR.

What the brief asks for (summary)

• One script test under scripts/ with its harness in packages/*/tests/, as row 190's tests are.
• Two configs, two data roots, two Consoles, one fake OP. Consoles start outside agent ancestry in a transient user unit. They never use the live stack's units, ports or sockets. The script removes its units on exit (trap).
• Test (design section 7 R7 gate): Jason's test identity, granted collaborator on one hub project, signs in, makes a thread, sends to the granted instance, gets the reply in the thread, is refused on another project and on board reads, is revoked, is refused again, and the run ends. The factory shows nothing of the hub.
• Mutants: grant project list widened to *; revocation claim end skipped.
• Gate: Filbert approves the manifest; the script test and every scripts/test-*.sh green on Sage's gate; no unit, port or socket of the script outlives the run. R8 (live window) stays row 209.

Open reads before design

• R4 and R6 candidates (the brief says it builds on their combined candidate). Confirm what they expose: the hub sign-in, grant and revocation paths, and the Console's board-read refusal.
• Row 190's test harness (several instances on one uid): how it starts units and cleans up.
• Transient user unit support on this host (systemd-run --user), and how the existing tests detect it.

Status

• 18:53Z: worktree created, brief read. No code yet.

Handoff 2026-10-23 19:10:09Z (Mos ruling Q245, mos-1503)

Row 296 moves to Lena (Codex seat, thread db27faca-85fb-4ca0-9fc7-78d740d3fa3a, branch lena/row296). Quackerjack stops here: no builds, no claim, no queue writes. This design is the handoff; Lena continues it.

Base: 45dca69f6. R4 landed as 2efd127ce (ancestor of the base, checked with merge-base --is-ancestor). R6 landed as d3059c2c3 (row 294). R7 is claimable once Darkwing's window ends (Sage, sage-3592).

What I verified by reading (not by running):

• packages/cli/src/access.mjs:65 reads system.federation?.peers ?? [] (the R6 seam).
• packages/bus/src/user-cli.mjs exists at base (the user-cli transport).
• Row 190's tests (packages/cli/tests/stack-instances.test.mjs, stack-release.test.mjs T56, stack-release-fixture.mjs) drive stand-in units and fixtures. They do not start real transient user units. Not a model for R7's Consoles.
• Real transient-unit launch pattern: packages/conversation/src/cohort.mjs (scopeAvailable() and ScopeLauncher, systemd-run --user --scope -p Delegate=yes --unit=...; MANAGER_ENV = XDG_RUNTIME_DIR, DBUS_SESSION_BUS_ADDRESS).
• Test-only spawn of systemd-run in packages/conversation/tests is the nearest precedent for start/stop cleanup.

What I did NOT verify (so treat as open): the hub sign-in, grant, revocation and board-read paths in R4/R6 code; the Console startup flags; whether the fake OP can be a local stand-in; the trap-based unit cleanup.

Design (proposal, not yet checked against code):

1. scripts/test-federation-acceptance.sh: the entry point. Creates a mktemp -d scratch root under /mnt/storage/scratch/tmp (never ~/.mosaic-dev, never the live bus). trap removes every unit it started and the scratch root. Refuses to run if a transient unit name collides.
2. Two configs (instance A = hub-side granted seat, instance B = collaborator's home), each with its own MOSAIC_CONFIG, data root and federation.self; federation.peers names the other.
3. One fake OP: a local stand-in that issues Jason's test identity. Its port and socket are per-run, not the live ones.
4. Two Consoles, each in its own transient user unit (systemd-run --user --unit=mosaic-r7--<A|B>); outside agent ancestry; ports picked per run.
5. Test flow (the R7 gate): identity -> grant on one hub project -> sign in -> thread -> send to granted instance -> reply in thread -> refused on another project -> refused on board reads -> revoke -> refused after revoke -> run ends. Factory output must show nothing of the hub.
6. Mutants: (a) grant project list widened to *, must fail the other-project refusal; (b) revocation claim end skipped, must fail the refusal-after-revoke check.

Unresolved decisions for Lena (I did not decide these):

• Whether the harness lives in packages/cli/tests (matching row 190) or packages/conversation/tests (matching cohort.mjs). Brief says packages/*/tests/; pick the one that owns the Console start path.
• Fake OP: in-process HTTP stand-in versus file-backed identity. No preference recorded.
• Port selection: fixed range per run vs OS-assigned. Brief forbids the live ports; it does not pick a range.

R2 and R5 learning:

• R2 (row 203, #1701) landed as a4da4e6df. I built it. Relevant to R7: validateFederation in scripts/mosaic-config.mjs rejects unknown keys and bad ids, and test-config.sh has an FED block (expect_fed_refusal). Use federation.self/peers exactly as row 203 validates them. Mutant runner pattern: one textual edit that must occur exactly once, then a named test must fail; restore from git. Name filter gotcha: node --test --test-name-pattern skips subtests whose parent does not match; include the parent name (T57 for row 203).
• R5: I did not build R5 in this session and have no first-hand notes on it. Ask Sage for R5's landing sha and notes.

Status: design written; no code, no test runs, no claim, no push of the test files. Next owner: Lena.

Lena continuation, 2026-10-23

Owner: Lena, thread db27faca-85fb-4ca0-9fc7-78d740d3fa3a. Branch lena/row296. Base remains 45dca69f6. Quackerjack's handoff c6ab9bdc9 preserved by cherry-pick ad84c1e45. Issue #1802 exists. Reviewer Filbert. Sage owns queue state, shared test slot and landing. Status: Untested. All Node, tests and credential-helper fetch/push held for Sage CLEAR (sage-3615). The earlier canonical queue next lena read finished with nothing and a rendered-view warning; no queue state was changed. Lena's owner registration is Sage's action.

Design decisions sent to Sage (lena-0003)

• Harness under packages/cli/tests, beside row 190's instance proof; shell entry under scripts/.
• Existing HTTPS fakeOidc helper, its generated scratch CA trusted through NODE_EXTRA_CA_CERTS by each real Console CLI. One fake OP, two client ids; no TLS verification bypass.
• Kernel-assigned loopback ports (0), discovered from each Console's startup log. Public origins are distinct *.example.test HTTPS names; HTTP requests model the loopback reverse proxy with exact Host and Origin. This proves the Console boundary, not a deployed HTTPS front end (R8).
• Transient user service units, not scopes: a scope leaves ancestry beneath the caller; the real user proof walks ancestry. Each Console runs the existing cli.mjs serve --sso --send verb.
• Real startHost broker process and createLauncher. Only the harness adapter is a test stand-in, as in thread-revoke.test.mjs. A real sandboxed run replies through the broker.
• Scratch-only setup and access grant/revoke use real human CLI transport from a transient service. No injected principal verifier, direct bindHuman/bindUser, or simulated user transport.
• Every service has a unique per-run prefix and is recorded before launch. Shell EXIT/INT/TERM traps stop only these services, wait for collection and verify socket and port closure.
• Claim-end assertion reads append-only claim history independently of the launcher's final shutdown so the claim-end mutant cannot be hidden by endLaunch's cleanup.

Implementation plan

Goal: an executable scratch proof of federation section 7's R7 gate, without live state. Architecture: shell lifecycle owner, Node test driver, test-only host fixture, existing real Console CLI. Tech: Bash, Node ESM/node:test, systemd user transient services, existing fake OP and bwrap adapter. Spec: docs/plans/2026-10-23_mvp-readiness-rows.md, FED R7, plus preserved handoff above.

Constraints: scratch only under /mnt/storage/scratch/tmp; no live units/ports/sockets/configs or credential reads; no production behavior/policy changes; no old test edits without Mos via Sage.

Review focus: partial startup cleanup; signals; unit name collision; mismatched hub/factory sessions; claim end versus process exit. Each must have explicit assertions or a negative cleanup exercise.

• Write acceptance driver assertions: Jason test sub binds as collaborator (owner is another fixture identity), one allowed project/instance, thread send and correlated RESULT, private project creation/read refused, board reads refused, factory has no hub objects, revoke denies session and re-login, claim history records access-revoked and runner ends.
• Write shell lifecycle: scratch HOME/TMPDIR, unique unit ledger, bounded command timeout, traps stop/collect only this run's units, verify recorded sockets and ports no longer listen.
• Write host fixture using startHost(bootConfig), createLauncher with fakeHarnesses and the existing thread adapter. Prepare two named configs through bootstrap before fixture edits; supply only generated test client slots. Setup/grant/revoke through real human transport.
• On Sage CLEAR run only the new script, fix failures in new files and record exact results.
• Prepare explicit once-only mutant patches: R5 projects widened to ['*']; R4 #claimEnd call skipped. On Sage CLEAR prove each gives an assertion failure, restore bytes in finally.
• Publish manifest (paths, hashes, assertion mapping, old-test numstat), request Filbert review, push additive commits to lena/row296 only. Sage runs shared suites and gate and lands the row.

Pre-existing test changes: none (numstat: empty).

Source inspection corrections before first execution

• The real webui chat is channel-gated. Scratch businesses enable only harness pi and channel webui; the loader fixture re-exports the unchanged webui channel. The adapter replacement is exactly fakeHarnesses' pi.adapter export; no product file gains a fixture branch.
• startHost creates a real broker child and a real Unix proof socket; it is hosted in the test driver. The two Console services run the exact same-tree webui CLI, which is required by verifyPrincipal. Human setup runs the exact same-tree human-cli through humanTransport.
• The earlier in-process host proposal did not need a separate host fixture: startHost exposes the production launcher API directly, so the only helper file is federation-human.mjs.
• Read-only SQLite inspects claim evidence while the broker remains the sole writer. The test also checks the registered runner and engine pids/start times are gone after revocation.
• --kill-after-start kills the test process after both services have listened; Node teardown hooks cannot run. The shell trap owns service cleanup independently. This is an expected-red exercise; acceptance requires its kill marker, nonzero exit and successful cleanup line.
• Main driver environment is a whitelist (PATH, scratch HOME/TMPDIR, systemd manager transport, R7 run ids). Console/human services use env -i. No host preload or credential env is carried.
• No Node or test has run. Bash syntax and git diff --check passed during preparation.

Binding design correction: sage-3625, 2026-10-23 19:17:35Z

Sage rejected the transient-unit approach: "no transient systemd user units used to satisfy the human proof"; such a unit makes the ancestry walk miss the development agent, and creates host state under mos-1438. This direction replaces the brief/handoff's unit requirement for this build. No unit was created by the superseded draft, and no Node/test execution occurred.

Current design, replacing the earlier proposals above:

• Consoles are ordinary spawned child processes in their own process groups; no systemd commands, units, user manager or manager environment are involved.
• Test-only federation-host.mjs follows channel-fixture.mjs's mock.module pattern. It re-exports real readProcess and replaces verifyHuman's readProcess argument with real process fields plus ppid=1. This is explicitly a stand-in for Jason's shell, NOT a real ancestry acceptance proof. Real pid/uid/startTime/nonce/path checks still run; verifyPrincipal still checks the Console's actual direct parent path for user calls. No product source receives this override.
• Each fixture host runs real startBroker and createLauncher. Its local host API is the existing thread-revoke.test.mjs pattern, with only the pi adapter replaced via fakeHarnesses.
• Setup/grant/revoke are plain children using real CLI wrappers against that fixture proof seam.
• Shell records each detached group's leader pid and start time, checks both before signalling, and verifies no live group member, TCP listener or Unix listener remains. Killed-driver mode bypasses every Node teardown hook and still leaves cleanup to the shell trap.
• Read-only claim evidence and actual runner/engine exit remain independent assertions.
• Gate limitation: this proves the revised scratch integration, not real outside-agent ancestry, transient unit cleanup, TLS proxy deployment or live federation. R8 remains row 209.
• R5 landing found in git log: ddfb29af7 (#1700).
• Targeted Node/tests remain held. Sage permits the first lena/row296 push in the 19:41Z gap, with MOSAIC_GITEA_CREDENTIAL_FILE unset. 226 and 283 have earlier slots.

Current files: shell entry; federation-acceptance.test.mjs; fixtures/federation-host.mjs and fixtures/federation-human.mjs; this BUILD; mutant runner; manifest; append-only logs; preserved Quackerjack BUILD at ad84c1e45. No pre-existing test modification (numstat empty).

First candidate, 2026-10-23

Implementation is prepared under sage-3625, not yet executed. Static checks: Bash parser accepts scripts/test-federation-acceptance.sh; Python ast.parse accepts mutants.py; git diff --check clean. No test outcome is claimed. The root node_modules links the existing canonical install so the future targeted run needs no install or network. No webui editor build is involved.

M1 now requires the user project-list leak assertion (R7 ungranted project must not appear), so it proves exposure through the actual Console transport rather than only inspecting grant configuration. The private-project thread-create refusal and exact grant-list check follow it. M2 requires R7 revoke must end the claim as access-revoked before any host teardown.

Sage opened the push gap early in sage-3644 (19:23:04Z). The first candidate is committed as Untested, for push only to lena/row296 with MOSAIC_GITEA_CREDENTIAL_FILE unset. Node, acceptance, negative cleanup exercise and mutants still wait for the test slot. Full suites/gate are Sage's.

Filbert static review, filbert-296-first-push-static, 19:28:51Z

Review pinned f06a1f78c against 45dca69f6. Filbert verified all nine manifest hashes from Git objects, no product/pre-existing test changes, no systemd calls, fixture-only ancestry override, real path/uid/startTime/nonce checks, group ledger identity checks and child-only CA trust. He ran nothing. The verdict remains pending execution and gate evidence.

• C1 confirmed by inspection: subprocess.run(timeout=280) kills the shell on timeout and can bypass its trap, while the shell's own deadline plus cleanup can exceed 280 seconds. Removed the outer Python timeout (the simpler alternative Filbert explicitly allowed). Python now waits for the shell's bounded run and EXIT/TERM cleanup. Source restoration remains in finally.
• N1 retained limitation: a SIGKILL between spawn and the pid/startTime ledger append can leave an unrecorded group. The killed-driver exercise begins after both Consoles are recorded and does not cover that gap. Cleanup claims apply to recorded groups; no universal crash-proof process creation claim is made.
• N2 addressed before the mutant run: the named readonly claim-history assertion now runs after revoke and before waiting for the run's end. A skipped #claimEnd therefore cannot first fail the run-end wait. The final latest-op check still runs after engine/runner exit. If M2 instead fails an earlier condition, its runner must say NOT KILLED; diagnose the first failure on the same pinned candidate, retain the intended assertion, and rerun on Sage CLEAR. Never accept a timeout or change the mutant's expected assertion merely to mark it killed.
• N3 deferred cosmetic fixture detail: wrap() returns fn(value) without await, so async rejection skips its code mapping. This is not an authorization or cleanup change; the rejection still propagates, and no verdict relies on that fixture mapping.
• N4: corrected the exact-grant assertion's indentation in the new test.
• N5: prepare() first calls bootstrap --instance, then overwrites that bootstrap-generated config inside the unique disposable scratch HOME with fixture Console/provider settings. This is fixture-only setup, not a product/bootstrap overwrite path or a live-config operation. There remains one selected config per scratch instance, with the same marked data root.

Still Untested. No Node, acceptance, mutant, negative cleanup or CLI suite run. Static validation only; execution waits for Sage's test window. Follow-up push needs a currently authorized gap.

Push receipt and N2 ordering proof, 2026-10-23 19:31Z

Sage sage-3659 authorized push before 19:38Z. e855f82e18 was pushed to origin/lena/row296 with MOSAIC_GITEA_CREDENTIAL_FILE unset. No Node/tests ran.

N2, filbert-296-c1-ack: this path is synchronous after a successful reload. accessCommand awaits reload at access.mjs:289; reload awaits the access.reload transport at :297. broker.mjs:4645 calls #revokeLost before constructing its reply. #revokeLost (:1025–1034) calls #claimEnd inside Store.transaction, which executes COMMIT before returning (store.mjs:282–302). server.mjs:160–161 calls broker.request and then writes the response. Therefore the claim record is committed before the successful CLI call returns; no asynchronous sweep is required for this assertion. A refused reload can be reported by the CLI with exit 0, but then immediate revocation evidence is absent and this test must fail; that is not a race the test should hide. No polling change is needed.

T3 package comparison (sage-3659): the four planned runs execute only the new dedicated federation-acceptance.test.mjs and its fixture host/human children. They do not execute stack-copy, stack-release or the whole CLI suite. Existing helpers are imported; no old test module is run. Asked Researcher to confirm against row216. Full CLI suite remains a later Sage gate requirement, outside the targeted T3 command list. All test resources are unique scratch state.

R2 rebase and slot start, 2026-10-24, sage-4035/4036/4038

New branch lena/row296-r2, base e0c6142f3a. Original branch remains a1321f8f9; no force push. Rebased tip before evidence: 17e421cf4. Only conflicts were append-only BUILD-LOG.md and docs/SESSIONS.md: retained all new-base bytes and appended this row's original suffix. All other row files byte-identical to a1321f8f9. Range-diff receipt: run-r2/range-diff.txt. Patch IDs: run-r2/patch-ids.txt. Mapping: ad84c1e45=4a16ebaa4, f06a1f78c→6418b1b12 (only append-log context), e855f82e1=bbc38067f, a1321f8f9=17e421cf4. Stable patch IDs match for 1/3/4; commit 2 differs because patch-id includes changed append-log context. No source delta.

The prior local root node_modules symlink was unlinked without following it. Own-worktree root npm ci --ignore-scripts --no-audit --no-fund: exit 0, 128 packages. Own-worktree webui same command: exit 0, 2 packages. Both node_modules are real local directories. Canonical dependencies were never installed, removed or changed. This supersedes the earlier link-based setup note.

Root install had completed before sage-4036 arrived. Exact queue command from worktree refused linked-worktree policy; reported as lena-0013 and held tests. Sage-4038 authorized queue verbs only in canonical; no npm/rm there. All further tests run in this worktree under short scratch TMPDIR, serially: plain script, killed-driver negative exercise, M1, M2, touched new test whole. Filbert cleared C1 on a1321f8f9 (filbert-296-c1-cleared); rebase preserves those source bytes.

Scoped acceptance results, 2026-10-24 09:55–09:57Z

Queue start succeeded: lena-296-start rev3888, briefed→in-progress, through the authorized canonical queue route. Earlier retries refused Sage assignment locks; no lock bypass occurred.

Commands ran serially in this worktree, each with a newly created six-character external TMPDIR under /mnt/storage/scratch/tmp and timeout -k 10. The script itself additionally creates its own unique r7.* root and owns child group cleanup. All five external TMPDIRs were empty and removed afterward (run-r2/tmpdir-cleanup.txt). No product/test source changed to obtain these results.

Run	Exit	Node pass/fail/skip/cancel	Evidence
plain script, timeout 600	0	1/0/0/0	run-r2/plain-1.log, 3.788 s Node duration
--kill-after-start, timeout 600	1 expected; negative wrapper 0	0/1/0/0 expected SIGKILL	run-r2/killed-1.log, both-Consoles-started marker and cleanup line
M1 driver, timeout 600	0 (script 1 expected)	0/1/0/0 expected mutation	run-r2/M1.log, KILLED by R7 ungranted project must not appear
M2 driver, timeout 600	0 (script 1 expected)	0/1/0/0 expected mutation	run-r2/M2.log, KILLED by R7 revoke must end the claim as access-revoked
new test whole without script context, timeout 60	0	0/0/1/0	run-r2/whole-file.log, expected dedicated-script skip

Every script run printed exactly R7 cleanup: no child groups, ports or sockets remain; no units created. The killed-driver run exercised shell cleanup with no Node teardown hooks. M1 and M2 failed on assertions, not startup failure or timeout; the runner found the named assertion and cleanup marker. pre-mutants.sha256 checked after each mutation: access.mjs and broker.mjs both restored byte-for-byte. git diff against e0c6142f3 confirms neither product path changed.

No unexpected first-run red: plain acceptance passed on its first run. Only the deliberate killed and mutated runs failed. The ordinary-file skip is intentional; the plain script ran its full test without skips. Module-mocking experimental/deprecation warnings are fixture-only. No old test was run or changed, so no mos-1272 placement was needed. N3 and N6 remain optional diagnostics; N1's documented spawn-to-ledger SIGKILL gap remains outside the killed-driver scenario.

Scope per sage-4035: dedicated script, touched new test whole, both mutants and cleanup checks. No full CLI suite, unrelated package suite, webui build or other scripts/test-*.sh ran in this slot. Those broader earlier review/gate requirements remain for Sage's integration gate; no full gate claim is made here. R8/live acceptance, real ancestry proof and deployment remain out of scope.