Files
2026-11-06 05:03:52 -06:00

20 KiB
Raw Permalink Blame History

Request lifecycle acceptance

request-lifecycle.mjs is an opt-in submission/observation runner for a later lead-authorized isolated instance. It uses the existing owner process proof and the running production host. It does not start a host, launch a scripted role, select resources or a mode, approve a PRD, mint credentials or provision services.

Prepare a named development instance with its matching instance.json, an existing factory source, installed tracker board and scoped credentials, and real PM, CTO, chief, coder, reviewer and assistant routes. Start that instance through the supported entry points in the authorized window. The owner supplies an existing assistant thread. The runner requires an explicit instance, config, business, expected owner, request quotation and evidence output directory. --owner checks the configured identity; it does not authenticate anybody.

Example with deliberate placeholders (replace every uppercase value):

node packages/cli/live/request-lifecycle.mjs \
  --instance pomo797 --config /ABS/CONFIG.json --business BUSINESS --owner OWNER \
  --thread THREAD_UUID --submit --authorize-isolated-run \
  --request-file /ABS/pomodoro-request.json --evidence /ABS/EVIDENCE

The owner selects and confirms the exact displayed plan in the signed-in Console. Autonomous PRD approval remains with the distinct technical arbiter. Guided mode requires the owner's answers and exact-version approval. No owner relays messages, creates a task manually, or runs worker shell commands. The quotation in ../tests/fixtures/pomodoro-request.json preserves the owner's original wording and provenance; it is not a raw runtime event export. The new runtime event and message IDs are retained separately.

The runner retains each observed request revision and final task/review trails. On complete reviewed delivery it reports the owner-only Console download URL. Download there, then rerun observation with --request EVENT_UUID and --artifact /ABS/DOWNLOADED.html in place of --submit --thread THREAD_UUID. The runner checks the exact reviewed artifact length and SHA-256 and keeps a copy. Unknown outcomes and timeouts retain evidence and fail; they never trigger a replacement submission or approval. Output files are created exclusively.

For the lead's live acceptance, retain the original input/message, owner plan confirmation, active approved snapshot, tracker create/assignment, code commit, independent reviewer fetch/test record and verdict, closure, delivery receipt, chief/assistant notification and downloaded bytes. Check that their project, snapshot, task, commit, review and verdict IDs form the same chain. In an offline browser, check the approved proposal's work/break durations, start, pause/resume, reset, phase transition and cue after a user gesture, and no network requests or external assets. The defaults are agent proposals approved in the product PRD. Inspect real independent role/test receipts before declaring live acceptance.

The repository fixture runs a separate runtime process over the same persistent fake tracker and real Store/repositories after each crash. Its scripted models use shipped role authority, launch admission/limits, session manifests, real typed tool subprocesses and Git bundles; coder and reviewer run the committed Node acceptance test independently. It exercises the signed-in HTTP listener and Chromium. It does not demonstrate live model quality or deployment readiness.

Crash restart follows the existing operator recovery rule: there is no automatic stale writer-lock reclaim. The fixture kills and reaps only its own process group, verifies that exact PID is dead, records the stale lock's identity and removes only its lock/socket before restart. The database, WAL and SHM are preserved. A live operator must establish the same facts and preserve evidence before recovery; this harness does not change that production rule.

ui797 isolated acceptance support (#797)

These three owner-run tools support a separately provisioned test instance. They do not provision a business, tracker, provider proxy or service units. Keep all runtime/evidence directories outside this checkout, the config directory and the instance data root. Use owner-only directories (0700) and campaign files (0600). No command targets .53 or installs/starts production services implicitly. The test instance uses its own proxy; there is no shared-proxy attachment here.

Requirements: the repository's pinned Node dependencies, Node with node:sqlite, OpenSSL, Chromium, NSS certutil, and Linux Bubblewrap (/usr/bin/bwrap) with usable user, mount, network and PID namespaces on every browser host. All examples assume a provisioner already created/validated the named instance through scripts/bootstrap.sh --instance ui797, with a matching instance.json. Preserve the owner shell HOME and agent markers. For temporary test directories set TMPDIR using mktemp -d /mnt/storage/scratch/tmp/XXXXXX (never /tmp).

Local identity and TLS

In an approved setup shell, set CFG to the test config's absolute path and RUNROOT to an existing owner-only runtime/evidence parent. The config's parent must be named mosaic-ui797. The following creates a new identity directory, private two-day CA/server certificates, a private RS256 signing key and the 0600 slot <configDir>/secrets/mosaic-stack/ui797-local. Existing files refuse. It prints the JSON block to insert into the test config, without changing that config or printing the client secret:

node packages/cli/live/isolated-identity.mjs prepare \
  --instance ui797 --config "$CFG" --owner ui797-owner \
  --runtime "$RUNROOT/identity" --tls-port 17443

Add that exact console.sso block through the approved test-config setup step. The public origin is https://127.0.0.1:17443, issuer https://127.0.0.1:17443/oidc/, behind the TLS forwarder to Console port 17440. There is no identity listener on port 17442. Only ui797-agent-owner can be issued, displayed as ui797-owner. Authorization immediately redirects back with a random, single-use 60-second code, state and S256 PKCE; signed ID tokens last 300 seconds. There is no refresh grant, alternate subject or live identity-provider integration.

Run these as separate normally supervised test services, not Console children of a browser agent. Preserve the real production entry point and ancestry proof:

node packages/cli/live/isolated-identity.mjs serve --runtime "$RUNROOT/identity"
MOSAIC_CONFIG="$CFG" NODE_EXTRA_CA_CERTS="$RUNROOT/identity/ca.pem" \
  node packages/webui/src/cli.mjs serve --business mvp3-ui797 \
  --port 17440 --board http://127.0.0.1:17439 --sso --send

The identity service binds only 127.0.0.1:17443 and forwards non-OIDC paths to 127.0.0.1:17440 with the original Host/Origin/cookies. It adds no subject headers. The Console validates discovery, JWKS, PKCE, nonce, token signature and its normal session/owner rules. A missing bus, business or ancestry proof still refuses. No human/Console binding shortcut or TLS-verification disable flag is used. Trust ca.pem via NODE_EXTRA_CA_CERTS only in these test services, never globally. Prepare and serve require the literal ui797, a mosaic-ui797 config directory and the current matching data-root marker. Serve revalidates persisted metadata; the budget commands enforce the same binding. Certificates expire after two days; do not overwrite identity material on restart.

Step-by-step real browser CLI for tester agents

start creates an exclusive evidence directory and fresh profile, then starts a private Unix-socket daemon. Subsequent invocations reuse that daemon. Either set UI797_EVIDENCE or pass --evidence DIR on each command. Only the exact loopback origins http://127.0.0.1:17440 and https://127.0.0.1:17443 are accepted. Every page/subresource redirect must stay on the selected origin. Other targets and WebSocket connections outside that origin are blocked before the handshake by a forced local HTTP proxy with Chromium's loopback bypass disabled. It opens sockets only to the selected origin, refuses HTTP upgrades, and permits CONNECT only to the selected HTTPS host/port. Chromium and its TCP relay run in a mandatory Bubblewrap network namespace, with its own loopback up and no external interfaces or routes. The relay accepts only the forced proxy's TCP port and carries bytes over a 0600 Unix socket to that origin-checking host proxy. All other TCP and UDP, including IPv4/IPv6 STUN/TURN on other loopback addresses, stay inside the empty namespace. Bubblewrap's PID namespace and parent-death handling also contain its children. The WebRTC flag alone does not prevent UDP on Chromium 156; it remains defense in depth, alongside disabling QUIC. Request interception also guards redirects and subresources. A detected blocked network attempt makes the session fail until stop. The driver never contacts 7332 or mosaic.woltje.com.

The browser trusts the test CA through a private NSS database under its evidence directory. Bubblewrap starts with an empty mount view, adds read-only /usr, libraries and selected NSS/font/certificate configuration, a minimal private /dev and /proc, and private tmpfs /tmp, /run and home. Only this browser's evidence/profile/download directories and proxy/control sockets are writable host mounts. Host home, runtime directories, pathname sockets and credential paths are hidden. An explicit environment allowlist supplies a private HOME and XDG directories; host D-Bus/SSH-agent addresses and secrets are not inherited. The private NSS database is mounted at the sandbox HOME's .pki/nssdb; the owner shell HOME and host trust files remain unchanged. Certificate verification stays enabled; a different CA is rejected. Startup refuses when the required trust setup or namespace is unavailable; it does not fall back to a certificate-error exception.

Exact commands for a tester (replace discovered refs and button names with the current snapshot output):

export UI797_EVIDENCE="$RUNROOT/browser-a"
node packages/cli/live/ui-browser.mjs start \
  --evidence "$UI797_EVIDENCE" --origin https://127.0.0.1:17443 \
  --ca "$RUNROOT/identity/ca.pem"
node packages/cli/live/ui-browser.mjs goto '/#chat'
node packages/cli/live/ui-browser.mjs snapshot
node packages/cli/live/ui-browser.mjs click --text Chat
node packages/cli/live/ui-browser.mjs type r1-123 'Build a Pomodoro timer.'
node packages/cli/live/ui-browser.mjs press Enter
node packages/cli/live/ui-browser.mjs wait --text 'Choose how to proceed' --timeout 120
node packages/cli/live/ui-browser.mjs screenshot request-plan
node packages/cli/live/ui-browser.mjs click r1-456
node packages/cli/live/ui-browser.mjs downloads
node packages/cli/live/ui-browser.mjs stop

With a sole provider, the initial page load signs in immediately via the normal OIDC redirects. snapshot reports visible accessibility-tree text and interactive roles/names/disabled states, with document-scoped stable refs. Snapshot again after navigation; stale refs refuse. click --text needs exactly one enabled exact accessible-name match. type focuses with a real mouse click and inserts text at the current caret via CDP Input; it appends rather than clears. Supported keys: Enter, Tab, Escape, Backspace, Delete, Home/End, PageUp/Down, arrows, Space and letters/digits, with Ctrl/Control, Alt, Meta or Shift modifiers (for example press Ctrl+A, press Backspace to clear a focused field). Clicks use real CDP mouse input. No page-function calls or fetch/state mocks are injected. wait is bounded to 120 seconds per invocation. No uncertain write is replayed. A click that opens a native alert/confirm/prompt returns {dialog:{type,message}}; answer it with dialog accept or dialog dismiss (for example the Console's Resume confirmation). While a dialog is open every other command except stop refuses, and no screenshot is taken because the page is blocked; the receipt records the dialog type, message and answer.

Each accepted action, including failed commands, start and stop, writes NNNNN-before.png, NNNNN-after.png and one actions.jsonl receipt with result status. The journal omits URLs, query strings, text inputs, headers, bodies and cookies. Screenshots show page contents, so keep all evidence private and avoid entering secrets. Snapshots omit input values. Downloads are saved under browser GUIDs and listed with absolute paths, byte lengths and SHA-256; compare these with the product's delivery receipt. Partial downloads refuse. Profiles/evidence are retained on stop. An interrupted daemon is not automatically replaced or its commands replayed: inspect receipts and use a new evidence directory for a new explicitly authorized session. stop remains available after a refused network request. These tools do not yet open downloaded artifacts in an offline target.

Durable metered campaign stop

An owner creates a single 0600 campaign.json before launching inference:

{
  "version": 1,
  "instance": "ui797",
  "config": "/ABS/CONFIG/mosaic-ui797/config.json",
  "business": "mvp3-ui797",
  "owner": "ui797-owner",
  "start": "2026-11-06T02:00:00.000Z",
  "end": "2026-11-06T03:00:00.000Z",
  "caps": [
    {"provider":"proxy","route":"glm","model":"glm-5.3-flash","tokens":100000000},
    {"provider":"proxy","route":"anthropic","model":"claude-opus-5-5","tokens":10000000},
    {"provider":"proxy","route":"anthropic","model":"claude-sonnet-5-5","tokens":10000000}
  ]
}

Use the approved actual absolute config path, canonical UTC campaign start/end and all actual model routes; the dates above illustrate the schema. Caps are positive safe integers: at most 100,000,000 for exactly proxy/glm/glm-5.3-flash, and 10,000,000 for every other approved model. Lower caps allow zero-spend tests. Only proxy GLM, Claude (anthropic route, claude-*) and Codex (codex route, gpt-*) bindings are accepted. One provider/model has one allowance across all routes; conflicting bindings refuse. The campaign must end by 2026-11-07T06:00:00.000Z; earlier windows narrow that absolute cutoff, which checks also enforce independently. The test bus store must exist before initialization. Run the supervisor outside model-writable settings/data in a normally supervised owner service:

node packages/cli/live/budget-supervisor.mjs init \
  --campaign "$RUNROOT/campaign.json" --ledger "$RUNROOT/campaign-ledger"
node packages/cli/live/budget-supervisor.mjs serve \
  --campaign "$RUNROOT/campaign.json" --ledger "$RUNROOT/campaign-ledger" --interval 1

On-demand commands (safe for a tester to request; a triggered halt still requires the normal owner-service ancestry):

node packages/cli/live/budget-supervisor.mjs check \
  --campaign "$RUNROOT/campaign.json" --ledger "$RUNROOT/campaign-ledger"
node packages/cli/live/budget-supervisor.mjs status \
  --campaign "$RUNROOT/campaign.json" --ledger "$RUNROOT/campaign-ledger"

status prints persisted totals from the last successful accounting check versus each model's tokens cap, the end time and latch state. It does not refresh usage. The supervisor opens only the pinned test bus SQLite store read-only, using the same turnRecords accounting as token-limit (input + output + cacheWrite + cacheRead). Proxy telemetry is excluded. A run's immutable bundle manifest supplies its actual proxyRoute; fallback/drift/uncapped models refuse. Delivered/read messages and ended runs detect missing completed usage. An in-flight begin waits for its outcome; unknown/malformed completed usage stops the campaign.

Each provider/route/model has a separate append-only ledger, durable across rounds/restarts. Campaign bytes, config bytes, root identity and store identity are pinned; changed caps/epoch, rewritten/disappeared usage and damaged ledger files stop. Checks recompute each journal entry's tokens and totals from authoritative usage; a valid-JSON token mismatch durably latches and halts. Initialization exclusively writes campaign.json.binding.json beside the owner campaign, outside the ledger volume, before creating the ledger. Preserve this 0600 file and the campaign: they independently pin the target and ledger path when accounting storage is unavailable. Checks never recreate that file or an accounting epoch. The version-2 ledger manifest independently pins config bytes, so missing, unreadable, malformed or wrong-mode sidecars stop and still attempt halt against the verified target. Each report and halt receipt includes evidenceErrors; damaged files are preserved. An intact sidecar must agree with both the owner campaign bytes and ledger campaign/epoch. A running service retains its verified binding and epoch while revalidating the actual target on every tick. Legacy version-1 ledgers require their intact independent sidecar to verify config bytes; a sidecar-less legacy ledger refuses targeting. Never reinitialize or reset an existing campaign to work around missing evidence. Initialization is exclusive. There is no reset, raise or resume command. A stale check.lock after interruption requires owner diagnosis; never auto-reclaim it. An existing or uncreatable lock prevents accounting and still asserts the sticky halt against the verified ui797 binding on every check/serve tick. The lock is preserved, including when a stop was already latched. A changed target config refuses rather than halting an unverified data root. On campaign-instance-changed, the owner must diagnose the campaign's pin, binding (including root and store identity), config bytes and ledger target against the original evidence. Restore the original files and target identities if they can be verified, or end the campaign; there is no supported retarget or recovery command. Never reinitialize, reset or raise the existing campaign, or auto-reclaim its lock. A refused target is not proof of a halt or quiescence.

At any cap, deadline or accounting refusal, the supervisor fsyncs stop.json before invoking the existing mosaic halt --business ... --json production CLI with the selected test MOSAIC_CONFIG. The CLI's human-shell/ancestry proof stays intact. A refusal/unknown stop is retained as a failed halt-N.json receipt and check exits nonzero; the latch remains. Missing, renamed or nonprivate ledger directories still trigger halt using the independently validated owner binding. If storage cannot persist a latch or receipt, it explicitly reports stop-latch-not-durable and/or stop-receipt-not-durable after attempting halt; durable evidence then needs owner repair before any further campaign work. Do not strip markers, direct-bind a principal, write processing state, or treat that receipt as quiescence. A properly installed owner service must perform/confirm halt. serve stays running on check/halt/storage errors, reports each failed tick and reasserts halt on every tick, including after a modelled Resume. It retains an in-memory stop if storage subsequently recovers. Every attempt revalidates the target; invalid, changed or foreign bindings refuse targeting. It never resumes. Manual Resume is outside the campaign protocol.

This is a metered stop, not strict inference admission or a hard provider ceiling. Concurrent/in-flight work, transcript accounting and the 0.1–10-second check interval can overshoot a cap. Install the native token-limit controls as well, retain accounting evidence, and use only the lead-approved metered-stop acceptance envelope. These tools add no model-dispatch hook or network quota gate.

Focused verification (no model spend, no remote access):

TMPDIR=$(mktemp -d /mnt/storage/scratch/tmp/XXXXXX) \
  node --test --test-concurrency=1 packages/cli/live/*.test.mjs

Run these files sequentially: their integration fixtures exclusively bind the fixed test ports 17440/17443 and fail if another process owns them. The browser SSO test starts the unchanged production cli.mjs serve --sso without --send; it proves genuine sign-in, not authenticated broker writes or real-model quality. The budget tests inject a tiny cap and a fake halt at the supervisor boundary; they also prove the real CLI refuses an agent environment. No live business is halted by these tests.