diff --git a/BUILD-LOG.md b/BUILD-LOG.md index 0fc793057..1911f6310 100644 --- a/BUILD-LOG.md +++ b/BUILD-LOG.md @@ -6193,3 +6193,6 @@ Before: clean isolated checkout at e9271841f3af23bf2a72aa17d9d67a1ec5abe017; no After: personally authored three-file skills/ms-ux-improvement coordinator package, complementary to existing frontend execution guidance; final candidate-02 content manifest b93d7836ed7c70db62386192d6e1db5c4f867f841aaa7645c3c861def5a9b140. Retained 19 bounded clean-context behavioral samples / 186 raw files: six unguided baselines, seven candidate-01 samples including ready-review control, six candidate-02 samples. Candidate-01's two unnecessary frozen-method recovery conditions and already-correct control remain recorded; narrowed wording and final tests show no extra gate, without causal/statistical improvement claim. Fresh package-focused skill-creator/resource validation passed for workspace, frozen snapshot and scratch copy; patch/whitespace, arithmetic, integrity, unchanged baseline and append-only ownership checks passed. Entry 637 words (2,210 total) and one-model synthetic/procedural-scope limits retained as author concerns. Exact source, patch, hashes, commands, raw evidence and independent-review brief: /mnt/storage/scratch/mosaic-lead/skill-ux-improvement/author/run-20261105T022125Z/HANDOFF.md. Author task complete; independent source approval pending. No application suites, commits, forge/queue writes, publication, installation, runtime activation or protected actions. 2026-11-04 | #818 / Gitea #1845 canonical skill integration verified | Exact3 reviewed Markdown files match source hashes; canonical skill-creator validator and whitespace checks pass. Independent review APPROVE/no findings,14 focused checks and a fresh bounded forward-use handoff retained. All prior canonical and author log bytes preserved. Package is available in source by explicit path; no managed/personal installation, activation, publication or user-acceptance claim. MVP activation remains pinned to parentbd8120318, independent of this skill commit. + +2026-11-05 | #797 ui797 support implementation phase | Explicit lead charter from 05fdfef9ea1fdddf58c725c04f229ca8559fc054 on feat/ui797-support: local isolated OIDC/TLS, persistent CDP browser CLI and durable metered campaign supervisor only under packages/cli/live. Queue next refused linked worktree (exit 2); canonical queue/forge ownership remains with lead. Initial author tests identified explicit-config selection and browser evidence-count defects, corrected locally; real browser SSO and full focused gates in progress. No .53/live access, push, runtime/role/contract edits, attribution trailers or stash. +2026-11-05 | #797 ui797 support author candidate verified | Added local single-subject HTTPS OIDC and fixed TLS forwarder, persistent real-input Chromium CLI with private NSS trust, and external durable per-model metered-stop ledgers. Final focused run: 34 pass / 0 fail / 0 skip (identity 6, browser 5, budget 23); dependency regressions: 111 pass / 0 fail / 0 skip; supported processing CLI regressions: 4 pass / 0 fail / 0 skip. Initial configuration-selection, Basic credential decoding, browser readiness/count and shutdown issues were corrected; all final gates pass. Source remains confined to packages/cli/live plus append-only protocol records. Local commit authorized; independent exact-candidate review and supervised-service acceptance remain with lead. No .53/live activity, model spend, push or product-runtime/role/contract changes. Evidence: /mnt/storage/scratch/mosaic-lead/mvp3/ui797-support-r1-evidence/; report: /mnt/storage/scratch/mosaic-lead/mvp3/ui797-support-r1-report.md. diff --git a/docs/SESSIONS.md b/docs/SESSIONS.md index 3a717e1c2..4c93312ee 100644 --- a/docs/SESSIONS.md +++ b/docs/SESSIONS.md @@ -2064,3 +2064,5 @@ are never rewritten or removed; corrections are new entries. 2026-11-05 | codex-ux-skill-author | Vikunja #818 UX improvement skill author handoff | Candidate-02 three-file package personally authored; package/copy/patch/arithmetic/integrity checks passed; 19 synthetic behavioral samples and all 186 raw artifacts retained with candidate-01 issue/correction and limits. Exact manifest b93d7836ed7c70db62386192d6e1db5c4f867f841aaa7645c3c861def5a9b140; evidence /mnt/storage/scratch/mosaic-lead/skill-ux-improvement/author/run-20261105T022125Z/HANDOFF.md. Independent source review pending; no commit, app suites, forge/queue writes, installation, activation or publication. 2026-11-04 | Mos, skill integrator | #818 / Gitea #1845 | Integrated exact independently approved ms-ux-improvement candidate02 into source;3hashes/validator/whitespace pass and author histories preserved. Separate commit from MVP; its activation target staysbd8120318. No install, activation, publication or live action. +2026-11-05 | Codex ui797 implementation child | #797 isolated OIDC/TLS, browser CLI and campaign budget support from 05fdfef9e on feat/ui797-support | Author implementation/tests underway in lead-chartered worktree; queue refused linked worktree, no canonical queue writes or protected actions. +2026-11-05 | Codex ui797 implementation child | #797 feat/ui797-support author handoff | Three isolated test tools and focused tests ready for lead review; final unique gates 149 pass / 0 fail / 0 skip, real production-CLI browser OIDC sign-in included, metered-stop/ancestry limits documented. Local commit only; no remote/live activity, spend, role/contract or product runtime edits. diff --git a/packages/cli/live/README.md b/packages/cli/live/README.md index 205740c04..568d6637c 100644 --- a/packages/cli/live/README.md +++ b/packages/cli/live/README.md @@ -61,3 +61,218 @@ its own process group, verifies that exact PID is dead, records the stale lock's identity and removes only its lock/socket before restart. The database, WAL and SHM are preserved. A live operator must establish the same facts and preserve evidence before recovery; this harness does not change that production rule. + +## ui797 isolated acceptance support (#797) + +These three owner-run tools support a separately provisioned **test** instance. +They do not provision a business, tracker, provider proxy or service units. Keep +all runtime/evidence directories outside this checkout, the config directory and +the instance data root. Use owner-only directories (0700) and campaign files +(0600). No command targets .53 or installs/starts production services implicitly. +The test instance uses its own proxy; there is no shared-proxy attachment here. + +Requirements: the repository's pinned Node dependencies, Node with `node:sqlite`, +OpenSSL, Chromium, NSS `certutil`, and Linux Bubblewrap (`/usr/bin/bwrap`) when the +OS user's legacy `~/.pki/nssdb` exists. All examples assume a provisioner already +created/validated the named instance through `scripts/bootstrap.sh --instance +ui797`, with a matching `instance.json`. Never change HOME or clear agent markers. +For temporary test directories set `TMPDIR` using +`mktemp -d /mnt/storage/scratch/tmp/XXXXXX` (never `/tmp`). + +### Local identity and TLS + +In an approved setup shell, set `CFG` to the **test** config's absolute path and +`RUNROOT` to an existing owner-only runtime/evidence parent. The config's parent +must be named `mosaic-ui797`. The following creates a new identity directory, +private two-day CA/server certificates, a private RS256 signing key and the +0600 slot `/secrets/mosaic-stack/ui797-local`. Existing files refuse. +It prints the JSON block to insert into the test config, without changing that +config or printing the client secret: + +```sh +node packages/cli/live/isolated-identity.mjs prepare \ + --instance ui797 --config "$CFG" --owner ui797-owner \ + --runtime "$RUNROOT/identity" --tls-port 17443 +``` + +Add that exact `console.sso` block through the approved test-config setup step. +The public origin is `https://127.0.0.1:17443`, issuer +`https://127.0.0.1:17443/oidc/`. Only `ui797-agent-owner` can be issued, displayed +as `ui797-owner`. Authorization immediately redirects back with a random, +single-use 60-second code, state and S256 PKCE; signed ID tokens last 300 seconds. +There is no refresh grant, alternate subject or live identity-provider integration. + +Run these as separate normally supervised **test** services, not Console children +of a browser agent. Preserve the real production entry point and ancestry proof: + +```sh +node packages/cli/live/isolated-identity.mjs serve --runtime "$RUNROOT/identity" +``` + +```sh +MOSAIC_CONFIG="$CFG" NODE_EXTRA_CA_CERTS="$RUNROOT/identity/ca.pem" \ + node packages/webui/src/cli.mjs serve --business mvp3-ui797 \ + --port 17440 --board http://127.0.0.1:17439 --sso --send +``` + +The identity service binds only 127.0.0.1:17443 and forwards non-OIDC paths to +127.0.0.1:17440 with the original Host/Origin/cookies. It adds no subject headers. +The Console validates discovery, JWKS, PKCE, nonce, token signature and its normal +session/owner rules. A missing bus, business or ancestry proof still refuses. +No human/Console binding shortcut or TLS-verification disable flag is used. +Trust `ca.pem` via `NODE_EXTRA_CA_CERTS` only in these test services, never globally. +Certificates expire after two days; do not overwrite identity material on restart. + +### Step-by-step real browser CLI for tester agents + +`start` creates an exclusive evidence directory and fresh profile, then starts a +private Unix-socket daemon. Subsequent invocations reuse that daemon. Either set +`UI797_EVIDENCE` or pass `--evidence DIR` on each command. Only the exact loopback +origins `http://127.0.0.1:17440` and `https://127.0.0.1:17443` are accepted. Every +page/subresource redirect must stay on the selected origin. Other targets and +WebSockets are blocked; a blocked network attempt makes the session fail until +`stop`. The driver never contacts 7332 or mosaic.woltje.com. + +The browser trusts the test CA through a private NSS database under its evidence +directory. If Chromium would prefer the user's legacy NSS store, Bubblewrap +mounts the private database over that path **only inside the browser process's +mount namespace**. HOME and host trust files remain unchanged. Certificate +verification stays enabled; a different CA is rejected. Startup refuses when the +required trust setup or namespace is unavailable; it does not fall back to a +certificate-error exception. + +Exact commands for a tester (replace discovered refs and button names with the +current `snapshot` output): + +```sh +export UI797_EVIDENCE="$RUNROOT/browser-a" +node packages/cli/live/ui-browser.mjs start \ + --evidence "$UI797_EVIDENCE" --origin https://127.0.0.1:17443 \ + --ca "$RUNROOT/identity/ca.pem" +node packages/cli/live/ui-browser.mjs goto '/#chat' +node packages/cli/live/ui-browser.mjs snapshot +node packages/cli/live/ui-browser.mjs click --text Chat +node packages/cli/live/ui-browser.mjs type r1-123 'Build a Pomodoro timer.' +node packages/cli/live/ui-browser.mjs press Enter +node packages/cli/live/ui-browser.mjs wait --text 'Choose how to proceed' --timeout 120 +node packages/cli/live/ui-browser.mjs screenshot request-plan +node packages/cli/live/ui-browser.mjs click r1-456 +node packages/cli/live/ui-browser.mjs downloads +node packages/cli/live/ui-browser.mjs stop +``` + +With a sole provider, the initial page load signs in immediately via the normal +OIDC redirects. `snapshot` reports visible accessibility-tree text and interactive +roles/names/disabled states, with document-scoped stable refs. Snapshot again after +navigation; stale refs refuse. `click --text` needs exactly one enabled exact +accessible-name match. `type` focuses with a real mouse click and inserts text at +the current caret via CDP Input; it appends rather than clears. Supported keys: +Enter, Tab, Escape, Backspace, Delete, Home/End, PageUp/Down, arrows, Space and +letters/digits, with Ctrl/Control, Alt, Meta or Shift modifiers (for example +`press Ctrl+A`, `press Backspace` to clear a focused field). Clicks use +real CDP mouse input. No page-function calls or fetch/state mocks are injected. +`wait` is bounded to 120 seconds per invocation. No uncertain write is replayed. + +Each accepted action, including failed commands, start and stop, writes +`NNNNN-before.png`, `NNNNN-after.png` and one `actions.jsonl` receipt with result +status. The journal omits URLs, query strings, text inputs, headers, bodies and +cookies. Screenshots show page contents, so keep all evidence private and avoid +entering secrets. Snapshots omit input values. Downloads are saved under browser +GUIDs and listed with absolute paths, byte lengths and SHA-256; compare these +with the product's delivery receipt. Partial downloads refuse. Profiles/evidence +are retained on stop. An interrupted daemon is not automatically replaced or its +commands replayed: inspect receipts and use a new evidence directory for a new +explicitly authorized session. `stop` remains available after a refused network +request. These tools do not yet open downloaded artifacts in an offline target. + +### Durable metered campaign stop + +An owner creates a single 0600 `campaign.json` **before launching inference**: + +```json +{ + "version": 1, + "instance": "ui797", + "config": "/ABS/CONFIG/mosaic-ui797/config.json", + "business": "mvp3-ui797", + "owner": "ui797-owner", + "start": "2026-11-06T02:00:00.000Z", + "end": "2026-11-06T03:00:00.000Z", + "caps": [ + {"provider":"proxy","route":"glm","model":"glm-5.3-flash","tokens":10000000}, + {"provider":"proxy","route":"anthropic","model":"claude-opus-5-5","tokens":10000000}, + {"provider":"proxy","route":"anthropic","model":"claude-sonnet-5-5","tokens":10000000} + ] +} +``` + +Use the approved actual absolute config path, canonical UTC campaign start/end +and **all** actual model routes; the dates above illustrate the schema. Caps are +positive safe integers at most 10,000,000; tiny values allow zero-spend tests. +The test bus store must exist before initialization. Run the supervisor outside +model-writable settings/data in a normally supervised owner service: + +```sh +node packages/cli/live/budget-supervisor.mjs init \ + --campaign "$RUNROOT/campaign.json" --ledger "$RUNROOT/campaign-ledger" +node packages/cli/live/budget-supervisor.mjs serve \ + --campaign "$RUNROOT/campaign.json" --ledger "$RUNROOT/campaign-ledger" --interval 1 +``` + +On-demand commands (safe for a tester to request; a triggered halt still requires +the normal owner-service ancestry): + +```sh +node packages/cli/live/budget-supervisor.mjs check \ + --campaign "$RUNROOT/campaign.json" --ledger "$RUNROOT/campaign-ledger" +node packages/cli/live/budget-supervisor.mjs status \ + --campaign "$RUNROOT/campaign.json" --ledger "$RUNROOT/campaign-ledger" +``` + +`status` prints persisted totals from the last successful accounting check versus +each model's `tokens` cap, the end time and latch state. It does not refresh usage. +The supervisor opens only the pinned test bus SQLite store read-only, using the +same `turnRecords` accounting as token-limit (input + output + cacheWrite + +cacheRead). Proxy telemetry is excluded. A run's immutable bundle manifest supplies +its actual `proxyRoute`; fallback/drift/uncapped models refuse. Delivered/read +messages and ended runs detect missing completed usage. An in-flight begin waits +for its outcome; unknown/malformed completed usage stops the campaign. + +Each provider/route/model has a separate append-only ledger, durable across +rounds/restarts. Campaign bytes, root identity and store identity are pinned; +changed caps/epoch, rewritten/disappeared usage and damaged ledger files stop. +Initialization is exclusive. There is no reset, raise or resume command. A stale +`check.lock` after interruption requires owner diagnosis; never auto-reclaim it. +A changed target config refuses rather than halting an unverified data root. + +At any cap, deadline or accounting refusal, the supervisor fsyncs `stop.json` +before invoking the existing `mosaic halt --business ... --json` production CLI +with the selected test `MOSAIC_CONFIG`. The CLI's human-shell/ancestry proof stays +intact. A refusal/unknown stop is retained as a failed `halt-N.json` receipt and +exits nonzero; the latch remains. If storage cannot persist a latch or receipt, it +still attempts halt and reports the storage failure; durable evidence then needs +owner repair before any further campaign work. Do not strip markers, direct-bind a principal, +write processing state, or treat that receipt as quiescence. A properly installed +owner service must perform/confirm halt. `serve` reasserts a latched halt each tick; +it never resumes. Manual Resume is outside the campaign protocol. + +This is a **metered stop**, not strict inference admission or a hard provider +ceiling. Concurrent/in-flight work, transcript accounting and the 0.1–10-second +check interval can overshoot a cap. Install the native token-limit controls as +well, retain accounting evidence, and use only the lead-approved metered-stop +acceptance envelope. These tools add no model-dispatch hook or network quota gate. + +Focused verification (no model spend, no remote access): + +```sh +TMPDIR=$(mktemp -d /mnt/storage/scratch/tmp/XXXXXX) \ + node --test --test-concurrency=1 packages/cli/live/*.test.mjs +``` + +Run these files sequentially: their integration fixtures exclusively bind the +fixed test ports 17440/17443 and fail if another process owns them. The browser +SSO test starts the unchanged production `cli.mjs serve --sso` without `--send`; +it proves genuine sign-in, not authenticated broker writes or real-model quality. +The budget tests inject a tiny cap and a fake halt at the supervisor boundary; +they also prove the real CLI refuses an agent environment. No live business is +halted by these tests. diff --git a/packages/cli/live/budget-supervisor.mjs b/packages/cli/live/budget-supervisor.mjs new file mode 100644 index 000000000..6563d5596 --- /dev/null +++ b/packages/cli/live/budget-supervisor.mjs @@ -0,0 +1,206 @@ +#!/usr/bin/env node +// Metered stop, with accounting/in-flight delay. Never a provider-side hard ceiling. +import { DatabaseSync } from 'node:sqlite'; +import { createHash } from 'node:crypto'; +import { closeSync, existsSync, fsyncSync, lstatSync, mkdirSync, openSync, readdirSync, rmdirSync, writeFileSync } from 'node:fs'; +import { join, dirname } from 'node:path'; +import { spawnSync } from 'node:child_process'; +import { turnRecords } from '../../bus/src/usage.mjs'; +import { exclusive, fail, isMain, options, outsideRuntime, plainPath, privateDir, readPrivate, ROOT, testInstance } from './support-common.mjs'; +const kinds = ['input','output','cacheWrite','cacheRead']; +const digest = x => createHash('sha256').update(typeof x === 'string' ? x : JSON.stringify(x)).digest('hex'); +const identity = c => JSON.stringify([c.provider,c.route,c.model]); +const filename = c => digest(identity(c)) + '.jsonl'; +const canonicalTime = s => typeof s === 'string' && Number.isFinite(Date.parse(s)) && new Date(s).toISOString() === s; +function campaignAt(path) { + privateDir(dirname(path)); + const text = readPrivate(path), c = JSON.parse(text); + if (c.version !== 1 || c.owner !== 'ui797-owner' || !/^[a-z][a-z0-9-]{1,63}$/.test(c.business ?? '') || !canonicalTime(c.start) || !canonicalTime(c.end) || c.end <= c.start || + !Array.isArray(c.caps) || !c.caps.length || c.caps.length > 20) fail('campaign-invalid'); + const keys = new Set(); + for (const cap of c.caps) { + if (Object.keys(cap).sort().join(',') !== 'model,provider,route,tokens' || !['proxy','anthropic','zai','openai'].includes(cap.provider) || + !['glm','anthropic','codex'].includes(cap.route) || !/^[a-zA-Z0-9][a-zA-Z0-9._:/-]{0,127}$/.test(cap.model ?? '') || + !Number.isSafeInteger(cap.tokens) || cap.tokens < 1 || cap.tokens > 10000000 || keys.has(identity(cap))) fail('campaign-cap-invalid'); + keys.add(identity(cap)); + } + return { c, hash:digest(text) }; +} +function binding(c) { + const { system } = testInstance(c.instance,c.config); + const st = lstatSync(system.dataRoot), dbPath = join(system.dataRoot,'bus/bus.sqlite'); plainPath(dbPath); + const db = lstatSync(dbPath); + if (!db.isFile() || db.uid !== process.getuid()) fail('test-store-required'); + return { instance:c.instance,config:c.config,dataRoot:system.dataRoot,business:c.business,owner:c.owner,rootIdentity:[st.dev,st.ino],storeIdentity:[db.dev,db.ino] }; +} +function checkBinding(pinned, c) { + const b = binding(c); + if (JSON.stringify(b) !== JSON.stringify(pinned)) fail('campaign-instance-changed'); + return b; +} +function writeJournal(file, row) { + readPrivate(file); + const fd = openSync(file,'a'); + try { writeFileSync(fd,JSON.stringify(row)+'\n'); fsyncSync(fd); } finally { closeSync(fd); } +} +function flushDir(dir) { const fd = openSync(dir,'r'); try { fsyncSync(fd); } finally { closeSync(fd); } } +export function initialize({ campaign, ledger }) { + const { c,hash } = campaignAt(campaign), b = binding(c); + outsideRuntime(campaign,{dataRoot:b.dataRoot},c.config); + outsideRuntime(ledger,{dataRoot:b.dataRoot},c.config); privateDir(ledger,true); + for (const cap of c.caps) exclusive(join(ledger,filename(cap)),JSON.stringify({ version:1,cap })+'\n'); + exclusive(join(ledger,'campaign.json'),JSON.stringify({ version:1,hash,binding:b,campaign:c })+'\n'); + flushDir(ledger); + return { initialized:true,ledger }; +} +export function readAccounting({ dataRoot,business,start,end }) { + const db = new DatabaseSync(join(dataRoot,'bus/bus.sqlite'),{readOnly:true}); + try { + db.exec('BEGIN'); + if (db.prepare("SELECT value FROM meta WHERE key='schema_version'").get()?.value !== '3s') fail('test-store-schema-refused'); + const store = { all:(sql,...args) => db.prepare(sql).all(...args) }; + const records = turnRecords(store,business); + const raw = store.all("SELECT * FROM usage_records WHERE business=? AND source<>'@proxy/cliproxy-queue' ORDER BY seq",business); + const completed = store.all(`SELECT DISTINCT d.holder_run run,m.id turn,d.at FROM deliveries d JOIN messages m ON m.id=d.message + WHERE m.business=? AND d.op='read' AND d.holder_run IS NOT NULL AND d.at>=?`,business,start); + const ended = store.all("SELECT subject run,at FROM events WHERE business=? AND kind='session.ended' AND at>=?",business,start); + db.exec('COMMIT'); + const phases = new Map(raw.map(r => [JSON.stringify([r.run,r.turn,r.source]),r.phase])); + const endedRuns = new Set(ended.map(r=>r.run)); + for (const r of raw) if (!['begin','end'].includes(r.phase) || !['known','unknown'].includes(r.status)) fail('usage-malformed'); + const interrupted = store.all(`SELECT DISTINCT d.holder_run run,m.id turn FROM deliveries d JOIN messages m ON m.id=d.message JOIN events e ON e.subject=d.holder_run AND e.business=m.business AND e.kind='session.ended' WHERE m.business=? AND d.op='delivered' AND e.at>=?`,business,start); + for (const t of interrupted) if (!records.some(r=>r.run===t.run && r.turn===t.turn && r.status==='known')) fail('usage-missing'); + for (const r of records) { + if (!canonicalTime(r.at)) fail('usage-malformed'); + if (r.at < start) continue; + // An in-flight begin is not unknown COMPLETED usage. It is checked on completion/end. + if (phases.get(JSON.stringify([r.run,r.turn,r.source])) !== 'end' && !endedRuns.has(r.run) && !completed.some(x=>x.run===r.run && x.turn===r.turn)) continue; + if (r.status !== 'known') fail('usage-unknown'); + } + const selected = records.filter(r => r.at >= start && r.status === 'known'); + for (const t of completed) if (!selected.some(r=>r.run===t.run && r.turn===t.turn)) fail('usage-missing'); + // Ended runs with begun turns must have a completed record for every source. + const rows = selected.map(r => { + if (!/^r[0-9a-f]{12}$/.test(r.run)) fail('usage-run-invalid'); + const manifest = JSON.parse(readPrivate(join(dataRoot,'launches',business,r.run,'bundle/manifest.json'))); + if (manifest.business !== business || manifest.run !== r.run || manifest.instance !== r.instance || manifest.provider !== r.provider || manifest.model !== r.model || manifest.route !== 'primary') fail('usage-route-drift'); + if (!['glm','anthropic','codex'].includes(manifest.proxyRoute)) fail('usage-route-unknown'); + return { ...r,route:manifest.proxyRoute }; + }); + return rows; + } finally { db.close(); } +} +export function supportedHalt(b, env=process.env) { + // This supervisor must be installed as a genuine owner service. Never strip agent markers, + // bindHuman/bindConsole, forge ancestry, or write control/processing.json directly. + const p = spawnSync(process.execPath,[join(ROOT,'packages/cli/src/cli.mjs'),'halt','--business',b.business,'--json'],{ + env:{...env,MOSAIC_CONFIG:b.config},encoding:'utf8',timeout:40000,maxBuffer:1024*1024, + }); + if (p.status !== 0) fail('supported-halt-refused-or-unknown'); + let result; try { result=JSON.parse(p.stdout); } catch { fail('supported-halt-invalid-reply'); } + if (result.halted !== true || result.quiescent !== true) fail('supported-halt-not-quiescent'); + return { halted:true,quiescent:true,generation:result.generation }; +} +function journal(ledger,cap) { + const text = readPrivate(join(ledger,filename(cap))); + if (!text.endsWith('\n')) fail('ledger-truncated'); + const [header,...entries] = text.trimEnd().split('\n').map(JSON.parse); + if (header.version !== 1 || JSON.stringify(header.cap) !== JSON.stringify(cap)) fail('ledger-header-changed'); + const seen = new Map(); let total = 0; + for (const e of entries) { + if (typeof e.key !== 'string' || typeof e.digest !== 'string' || !Number.isSafeInteger(e.tokens) || e.tokens < 0 || seen.has(e.key)) fail('ledger-malformed'); + seen.set(e.key,e); total += e.tokens; + if (!Number.isSafeInteger(total)) fail('ledger-overflow'); + } + return { entries:seen,total }; +} +function loadLedger(ledger) { + privateDir(ledger); const p=JSON.parse(readPrivate(join(ledger,'campaign.json'))); + if (p.version !== 1 || !p.binding || !p.campaign) fail('ledger-manifest-invalid'); + return p; +} +export async function check({ campaign,ledger }, { now=Date.now, read=readAccounting, halt=supportedHalt }={}) { + let p, integrityFailure=false; + try { p=loadLedger(ledger); } catch { + // With a damaged manifest, only the owner campaign and validated TEST marker may identify + // the target. Never silently rebuild the accounting epoch or continue with a zero ledger. + privateDir(ledger); const {c,hash}=campaignAt(campaign); + p={campaign:c,hash,binding:binding(c)}; integrityFailure=true; + } + const c=p.campaign; + // Pin the target before any halt. A changed config cannot redirect the stop into live data. + checkBinding(p.binding,c); outsideRuntime(ledger,{dataRoot:p.binding.dataRoot},c.config); + outsideRuntime(campaign,{dataRoot:p.binding.dataRoot},c.config); + const lock=join(ledger,'check.lock'); mkdirSync(lock,{mode:0o700}); + try { + let reason=integrityFailure?'ledger-manifest-invalid':null, totals=[]; + const stopFile=join(ledger,'stop.json'); + if (existsSync(stopFile)) { + reason='stop-already-latched'; + try { JSON.parse(readPrivate(stopFile)); } catch { reason='stop-latch-invalid'; } + } + try { + if (campaignAt(campaign).hash !== p.hash) fail('campaign-changed'); + const clock=now(); if (clock < Date.parse(c.start)) fail('campaign-not-started'); if (clock>=Date.parse(c.end)) fail('campaign-ended'); + if (!reason) { + const records=await read({...p.binding,start:c.start,end:c.end}), observed=new Set(); + const accounts=c.caps.map(cap=>({cap,...journal(ledger,cap)})); + for (const r of records) { + if (!canonicalTime(r.at) || r.at!Number.isSafeInteger(r[k]) || r[k]<0)) fail('usage-unknown-or-malformed'); + const a=accounts.find(a=>identity(a.cap)===identity(r)); if (!a) fail('usage-uncapped-route-model'); + const key=JSON.stringify([r.run,r.turn,r.source]); if (observed.has(key)) fail('usage-duplicate'); observed.add(key); + const tokens=kinds.reduce((sum,k)=>sum+r[k],0); if (!Number.isSafeInteger(tokens)) fail('usage-overflow'); + const hash=digest(r), previous=a.entries.get(key); + if (previous && previous.digest !== hash) fail('usage-rewritten'); + if (!previous) { + const e={key,digest:hash,tokens}; writeJournal(join(ledger,filename(a.cap)),e); a.entries.set(key,e); a.total+=tokens; + if (!Number.isSafeInteger(a.total)) fail('ledger-overflow'); + } + } + for (const a of accounts) for (const key of a.entries.keys()) if (!observed.has(key)) fail('usage-disappeared'); + totals=accounts.map(a=>({...a.cap,total:a.total})); + if (totals.some(a=>a.total>=a.tokens)) reason='model-cap-reached'; + } + } catch(e) { reason=/^[a-z-]{1,80}$/.test(e.message)?e.message:'accounting-failed'; } + if (reason) { + let storageError=null; + try { + if (!existsSync(stopFile)) { exclusive(stopFile,JSON.stringify({version:1,at:new Date(now()).toISOString(),reason,binding:p.binding,totals})+'\n'); flushDir(ledger); } + } catch { storageError='stop-latch-not-durable'; } + // Always attempt halt, even when the evidence volume cannot persist the latch. + let result,error; + try { checkBinding(p.binding,c); result=await halt(p.binding); } catch { error='supported-halt-refused-or-unknown'; } + try { + const attempt=readdirSync(ledger).filter(n=>/^halt-[0-9]+\.json$/.test(n)).length+1; + exclusive(join(ledger,`halt-${attempt}.json`),JSON.stringify({at:new Date(now()).toISOString(),ok:!error,result:result??null,error:error??null,storageError})+'\n'); flushDir(ledger); + } catch { storageError='stop-receipt-not-durable'; } + if (storageError) fail(storageError); + if (error) fail(error); + return {stopped:true,reason,totals,halt:result}; + } + return {stopped:false,totals}; + } finally { rmdirSync(lock); } +} +export function status({ campaign,ledger }) { + const p=loadLedger(ledger); checkBinding(p.binding,p.campaign); + outsideRuntime(campaign,{dataRoot:p.binding.dataRoot},p.campaign.config); + if (campaignAt(campaign).hash!==p.hash) fail('campaign-changed'); + const totals=p.campaign.caps.map(cap=>({...cap,total:journal(ledger,cap).total})); + return { stopped:existsSync(join(ledger,'stop.json')),end:p.campaign.end,totals }; +} +export async function main(argv) { + const command=argv.shift(),o=options(argv,['--campaign','--ledger','--interval']); + const opts={campaign:o['--campaign'],ledger:o['--ledger']}; + if (command==='init') return initialize(opts); + if (command==='status') return status(opts); + if (command==='check') return check(opts); + if (command!=='serve') fail('use init, check, serve or status'); + const interval=Number(o['--interval']??1); if (!Number.isFinite(interval) || interval<0.1 || interval>10) fail('interval must be 0.1..10 seconds'); + let running=true; for(const signal of ['SIGTERM','SIGINT']) process.once(signal,()=>{running=false;}); + while(running) { + // Reassert a latched halt on EVERY tick, so a later manual Resume cannot clear this campaign. + const result=await check(opts); console.log(JSON.stringify(result)); + await new Promise(r=>setTimeout(r,interval*1000)); + } +} +if (isMain(import.meta.url)) main(process.argv.slice(2)).then(r=>{if(r!==undefined)console.log(JSON.stringify(r,null,2));}).catch(e=>{console.error(e.message);process.exitCode=1;}); diff --git a/packages/cli/live/budget-supervisor.test.mjs b/packages/cli/live/budget-supervisor.test.mjs new file mode 100644 index 000000000..4aa922ab1 --- /dev/null +++ b/packages/cli/live/budget-supervisor.test.mjs @@ -0,0 +1,122 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdirSync,writeFileSync,readFileSync,readdirSync,unlinkSync,symlinkSync } from 'node:fs'; +import { join } from 'node:path'; +import { initialize,check,status,readAccounting,supportedHalt } from './budget-supervisor.mjs'; +import { fixture } from './test-support.mjs'; +import { Store } from '../../bus/src/store.mjs'; +const NOW=Date.parse('2026-11-06T02:30:00.000Z'); +const row=(extra={})=>({at:'2026-11-06T02:29:00.000Z',run:'r123456789abc',turn:'t1',source:'transcript',instance:'assistant',provider:'proxy',route:'glm',model:'glm-5.3-flash',status:'known',input:1,output:1,cacheWrite:1,cacheRead:1,...extra}); +function setup(t,tokens=5) { + const f=fixture(t),store=new Store(f.dataRoot);t.after(()=>store.close()); + const campaign=join(f.root,'campaign.json'),ledger=join(f.root,'ledger'); + const c={version:1,instance:'ui797',config:f.config,business:'mvp3-ui797',owner:'ui797-owner',start:'2026-11-06T02:00:00.000Z',end:'2026-11-06T03:00:00.000Z',caps:[{provider:'proxy',route:'glm',model:'glm-5.3-flash',tokens},{provider:'proxy',route:'anthropic',model:'claude-opus-5-5',tokens:6}]}; + writeFileSync(campaign,JSON.stringify(c),{mode:0o600});initialize({campaign,ledger}); + const halts=[],halt=b=>{halts.push(b);return {halted:true,quiescent:true,generation:'test'};}; + return {...f,store,campaign,ledger,c,halts,halt,opts:{campaign,ledger}}; +} +test('tiny cap counts all four token kinds per model, persists/deduplicates across checks and latches entire campaign',async t=>{ + const f=setup(t);const opts={now:()=>NOW,halt:f.halt,read:()=>[row()]}; + assert.equal((await check(f.opts,opts)).stopped,false); + assert.equal((await check(f.opts,opts)).totals[0].total,4);assert.equal(f.halts.length,0); + const stopped=await check(f.opts,{...opts,read:()=>[row(),row({turn:'t2',input:0,output:1,cacheRead:0,cacheWrite:0})]}); + assert.equal(stopped.reason,'model-cap-reached');assert.equal(stopped.totals[0].total,5);assert.equal(stopped.totals[1].total,0); + assert.equal(f.halts[0].dataRoot,f.dataRoot); + assert.equal(status(f.opts).stopped,true);assert.equal(status(f.opts).totals[0].total,5); + assert.equal((await check(f.opts,opts)).reason,'stop-already-latched');assert.equal(f.halts.length,2); + assert.throws(()=>initialize(f.opts)); +}); +for(const [name,records,reason] of [ + ['unknown',[row({status:'unknown'})],'usage-unknown-or-malformed'], + ['malformed',[row({input:-1})],'usage-unknown-or-malformed'], + ['unsafe integer',[row({input:Number.MAX_SAFE_INTEGER+1})],'usage-unknown-or-malformed'], + ['route drift',[row({route:'anthropic'})],'usage-uncapped-route-model'], + ['unlisted model',[row({model:'another-model'})],'usage-uncapped-route-model'], + ['duplicate',[row(),row()],'usage-duplicate'], +])test(`${name} latches before the test-only halt`,async t=>{ + const f=setup(t); + const result=await check(f.opts,{now:()=>NOW,read:()=>records,halt:b=>{assert.ok(readFileSync(join(f.ledger,'stop.json')).length);return f.halt(b);}}); + assert.equal(result.reason,reason);assert.equal(f.halts.length,1); +}); +test('campaign deadline, immutable owner cap and corrupt/missing ledger all stop without auto reset',async t=>{ + for(const variant of ['deadline','raise','malformed','missing']) { + const sub=await t.test(variant,async t=>{ + const f=setup(t);let now=NOW; + if(variant==='deadline')now=Date.parse(f.c.end); + if(variant==='raise'){f.c.caps[0].tokens=10;writeFileSync(f.campaign,JSON.stringify(f.c));} + if(variant==='malformed')writeFileSync(join(f.ledger,readdirSync(f.ledger).find(n=>n.endsWith('.jsonl'))),'{'); + if(variant==='missing')unlinkSync(join(f.ledger,readdirSync(f.ledger).find(n=>n.endsWith('.jsonl')))); + const result=await check(f.opts,{now:()=>now,read:()=>[],halt:f.halt});assert.equal(result.stopped,true);assert.equal(f.halts.length,1); + }); + } +}); +test('halt refusal keeps the durable stop and records an unknown halt, with no resume or raise',async t=>{ + const f=setup(t,1); + await assert.rejects(check(f.opts,{now:()=>NOW,read:()=>[row()],halt:()=>{throw new Error('human-required');}}),/supported-halt-refused/); + assert.equal(status(f.opts).stopped,true); + assert.equal(JSON.parse(readFileSync(join(f.ledger,'halt-1.json'))).ok,false); + await check(f.opts,{now:()=>NOW,halt:f.halt});assert.equal(f.halts.length,1); +}); +test('usage removal or rewrite is refused rather than refilling or double-counting',async t=>{ + const f=setup(t,100);await check(f.opts,{now:()=>NOW,read:()=>[row()],halt:f.halt}); + const result=await check(f.opts,{now:()=>NOW,read:()=>[row({input:0})],halt:f.halt});assert.equal(result.reason,'usage-rewritten'); + const g=setup(t,100);await check(g.opts,{now:()=>NOW,read:()=>[row()],halt:g.halt}); + assert.equal((await check(g.opts,{now:()=>NOW,read:()=>[],halt:g.halt})).reason,'usage-disappeared'); +}); +function insert(f,phase='end',status='known',source='transcript') { + f.store.run(`INSERT INTO usage_records(at,business,run,instance,provider,model,turn,source,phase,status,input,output,cache_write,cache_read) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?)`, + row().at,f.c.business,row().run,'assistant','proxy','glm-5.3-flash','t1',source,phase,status,...(status==='known'?[1,1,1,1]:[null,null,null,null])); + const manifest=join(f.dataRoot,'launches',f.c.business,row().run,'bundle');mkdirSync(manifest,{recursive:true,mode:0o700}); + writeFileSync(join(manifest,'manifest.json'),JSON.stringify({business:f.c.business,run:row().run,instance:'assistant',provider:'proxy',model:'glm-5.3-flash',proxyRoute:'glm',route:'primary'}),{mode:0o600}); +} +test('actual read-only bus store source matches token-limit records and excludes proxy telemetry',async t=>{ + const f=setup(t,100);insert(f);insert(f,'end','known','@proxy/cliproxy-queue'); + const r=readAccounting({dataRoot:f.dataRoot,business:f.c.business,start:f.c.start,end:f.c.end});assert.equal(r.length,1);assert.equal(r[0].route,'glm'); + assert.equal((await check(f.opts,{now:()=>NOW,halt:f.halt})).totals[0].total,4); + assert.equal(f.store.get('SELECT count(*) n FROM usage_records').n,2); +}); +test('a completed message with no usage records stops; an in-flight begin waits, an ended unknown turn stops',async t=>{ + const f=setup(t,100);insert(f,'begin'); + assert.equal((await check(f.opts,{now:()=>NOW,halt:f.halt})).stopped,false); + f.store.run(`INSERT INTO events(id,at,business,kind,subject,body) VALUES(?,?,?,?,?,?)`,'ended',row().at,f.c.business,'session.ended',row().run,'{}'); + assert.equal((await check(f.opts,{now:()=>NOW,halt:f.halt})).reason,'usage-unknown'); + const g=setup(t,100); + g.store.run(`INSERT INTO messages(id,at,business,from_role,from_run,to_role,class,body) VALUES(?,?,?,?,?,?,?,?)`,'t1',row().at,g.c.business,'chief','chief-run','assistant','note','work'); + g.store.run(`INSERT INTO deliveries(message,at,op,holder_run) VALUES(?,?,?,?)`,'t1',row().at,'read',row().run); + assert.equal((await check(g.opts,{now:()=>NOW,halt:g.halt})).reason,'usage-missing'); +}); +test('completed unknown source and fallback manifest fail closed against the real store',async t=>{ + const f=setup(t,100);insert(f,'end','unknown'); + assert.equal((await check(f.opts,{now:()=>NOW,halt:f.halt})).reason,'usage-unknown'); + const g=setup(t,100);insert(g); + const path=join(g.dataRoot,'launches',g.c.business,row().run,'bundle/manifest.json'),m=JSON.parse(readFileSync(path));m.route='fallback';writeFileSync(path,JSON.stringify(m)); + assert.equal((await check(g.opts,{now:()=>NOW,halt:g.halt})).reason,'usage-route-drift'); +}); +test('changed target config never directs a halt to a different root',async t=>{ + const f=setup(t,1),other=join(f.root,'other');mkdirSync(other);writeFileSync(join(other,'instance.json'),JSON.stringify({instanceVersion:1,name:'ui797'})); + const config=JSON.parse(readFileSync(f.config));config.dataRoot=other;writeFileSync(f.config,JSON.stringify(config)); + await assert.rejects(check(f.opts,{now:()=>NOW,read:()=>[row()],halt:f.halt}));assert.equal(f.halts.length,0); +}); +test('normal supported halt refuses an agent environment, preserving its markers',t=>{ + const f=setup(t);assert.throws(()=>supportedHalt({config:f.config,dataRoot:f.dataRoot,business:f.c.business},{...process.env,MOSAIC_AGENT_RUN:'test-agent'}),/supported-halt-refused/); +}); +test('a corrupt ledger manifest stops with the validated owner campaign, never recreates accounting',async t=>{ + const f=setup(t,100);writeFileSync(join(f.ledger,'campaign.json'),'{'); + const result=await check(f.opts,{now:()=>NOW,read:()=>[],halt:f.halt});assert.equal(result.reason,'ledger-manifest-invalid');assert.equal(f.halts.length,1); + assert.equal(readFileSync(join(f.ledger,'campaign.json'),'utf8'),'{'); +}); +test('an invalid existing stop latch still invokes the supported test halt',async t=>{ + const f=setup(t,100);mkdirSync(join(f.ledger,'stop.json'),{mode:0o700}); + const result=await check(f.opts,{now:()=>NOW,read:()=>[],halt:f.halt});assert.equal(result.reason,'stop-latch-invalid');assert.equal(f.halts.length,1); +}); +test('model-writable campaign paths refuse initialization',t=>{ + const f=fixture(t),store=new Store(f.dataRoot);t.after(()=>store.close()); + const campaign=join(f.dataRoot,'campaign.json');writeFileSync(campaign,JSON.stringify({version:1,instance:'ui797',config:f.config,business:'mvp3-ui797',owner:'ui797-owner',start:'2026-11-06T02:00:00.000Z',end:'2026-11-06T03:00:00.000Z',caps:[{provider:'proxy',route:'glm',model:'glm-5.3-flash',tokens:1}]}),{mode:0o600}); + assert.throws(()=>initialize({campaign,ledger:join(f.root,'ledger')}),/outside/); +}); + +test('a failed durable-latch write still halts and returns an explicit storage failure',async t=>{ + const f=setup(t,1);symlinkSync(join(f.root,'missing-stop-target'),join(f.ledger,'stop.json')); + await assert.rejects(check(f.opts,{now:()=>NOW,read:()=>[row()],halt:f.halt}),/stop-latch-not-durable/); + assert.equal(f.halts.length,1);assert.equal(JSON.parse(readFileSync(join(f.ledger,'halt-1.json'))).storageError,'stop-latch-not-durable'); +}); diff --git a/packages/cli/live/isolated-identity.mjs b/packages/cli/live/isolated-identity.mjs new file mode 100644 index 000000000..7711e26d5 --- /dev/null +++ b/packages/cli/live/isolated-identity.mjs @@ -0,0 +1,111 @@ +#!/usr/bin/env node +// Deliberately fake identity, real browser code flow and production OIDC validation. +import { createHash, createPrivateKey, createPublicKey, randomBytes } from 'node:crypto'; +import { chmodSync, mkdirSync } from 'node:fs'; +import { join, dirname } from 'node:path'; +import { spawnSync } from 'node:child_process'; +import { createServer } from 'node:https'; +import { request } from 'node:http'; +import { rsaKey, jwt } from '../../webui/tests/helpers/fake-oidc.mjs'; +import { checkConsole } from '../../webui/src/sso-config.mjs'; +import { exclusive, fail, isMain, options, outsideRuntime, plainPath, privateDir, readPrivate, testInstance } from './support-common.mjs'; +export const SUBJECT = 'ui797-agent-owner', OWNER = 'ui797-owner'; +const ORIGIN = 'https://127.0.0.1:17443', ISSUER = `${ORIGIN}/oidc/`, CLIENT = 'mosaic-ui797', SLOT = 'ui797-local'; +export function prepare({ instance, config, owner, runtime, tlsPort }) { + if (owner !== OWNER || String(tlsPort) !== '17443') fail('only ui797-owner and test TLS port 17443 supported'); + const { system } = testInstance(instance, config); + outsideRuntime(runtime, system, config); + privateDir(runtime, true); // exclusive: never replace a previous identity or secret + const secretDir = join(dirname(config), 'secrets', 'mosaic-stack'); + plainPath(secretDir); mkdirSync(secretDir, { recursive: true, mode: 0o700 }); + const secret = randomBytes(32).toString('base64url'); + exclusive(join(secretDir, SLOT), secret + '\n'); + const openssl = args => { + const r = spawnSync('openssl', args, { cwd: runtime, encoding: 'utf8' }); + if (r.status !== 0) fail('test certificate generation failed'); + }; + // All intermediates stay in the exclusive private runtime, never the repository. + exclusive(join(runtime, 'extensions.cnf'), 'basicConstraints=critical,CA:FALSE\nkeyUsage=critical,digitalSignature,keyEncipherment\nextendedKeyUsage=serverAuth\nsubjectAltName=IP:127.0.0.1\n'); + openssl(['req','-x509','-newkey','rsa:2048','-nodes','-days','2','-subj','/CN=ui797 private test CA','-addext','basicConstraints=critical,CA:TRUE','-keyout','ca.key','-out','ca.pem']); + openssl(['req','-new','-newkey','rsa:2048','-nodes','-subj','/CN=127.0.0.1','-keyout','tls.key','-out','tls.csr']); + openssl(['x509','-req','-in','tls.csr','-CA','ca.pem','-CAkey','ca.key','-CAcreateserial','-days','2','-extfile','extensions.cnf','-out','tls.pem']); + for (const name of ['ca.key','ca.pem','tls.key','tls.pem','tls.csr','ca.srl']) chmodSync(join(runtime, name), 0o600); + const signing = rsaKey('ui797-rsa'); + exclusive(join(runtime, 'signing.pem'), signing.privateKey.export({ type: 'pkcs8', format: 'pem' })); + const console = checkConsole({ sso: { publicOrigin: ORIGIN, providers: [{ id: SLOT, issuer: ISSUER, clientId: CLIENT, clientSecret: { slot: SLOT }, algs: ['RS256'], subjects: [{ sub: SUBJECT, name: OWNER }] }] } }); + exclusive(join(runtime, 'identity.json'), JSON.stringify({ version: 1, instance, config, owner, console }) + '\n'); + return { console }; +} +export async function serve(runtime, { clock = Date.now } = {}) { + privateDir(runtime); + const state = JSON.parse(readPrivate(join(runtime, 'identity.json'))); + const { system } = testInstance(state.instance, state.config); + outsideRuntime(runtime, system, state.config); + if (state.version !== 1 || state.owner !== OWNER || JSON.stringify(state.console) !== JSON.stringify(checkConsole(state.console)) || state.console.sso.publicOrigin !== ORIGIN || state.console.sso.providers.length !== 1 || state.console.sso.providers[0].issuer !== ISSUER || state.console.sso.providers[0].id !== SLOT || state.console.sso.providers[0].clientId !== CLIENT || state.console.sso.providers[0].clientSecret.slot !== SLOT || JSON.stringify(state.console.sso.providers[0].subjects) !== JSON.stringify([{ sub: SUBJECT, name: OWNER }])) fail('test identity metadata refused'); + const secret = readPrivate(join(dirname(state.config), 'secrets/mosaic-stack', SLOT)).trim(); + const privateKey = createPrivateKey(readPrivate(join(runtime, 'signing.pem'))); + const jwk = { ...createPublicKey(privateKey).export({ format: 'jwk' }), kid: 'ui797-rsa', use: 'sig', alg: 'RS256' }; + const codes = new Map(); + const server = createServer({ key: readPrivate(join(runtime, 'tls.key')), cert: readPrivate(join(runtime, 'tls.pem')) + readPrivate(join(runtime, 'ca.pem')) }, (req, res) => { + res.setHeader('cache-control', 'no-store'); + const json = (status, body) => { res.writeHead(status, { 'content-type': 'application/json' }); res.end(JSON.stringify(body)); }; + if (req.headers.host !== '127.0.0.1:17443') return json(403, { error: 'test_host_required' }); + const url = new URL(req.url, ORIGIN); + const path = url.pathname; + if (!path.startsWith('/oidc/')) { + // Fixed loopback upstream, original public Host/Origin and browser cookies intact. + // No subject/authentication headers are manufactured here. + const upstream = request({ hostname: '127.0.0.1', port: 17440, method: req.method, path: req.url, headers: req.headers }, reply => { + res.writeHead(reply.statusCode, reply.headers); reply.pipe(res); + }); + upstream.setTimeout(30000, () => upstream.destroy()); + upstream.on('error', () => { if (!res.headersSent) res.writeHead(502); res.end(); }); + req.on('aborted', () => upstream.destroy()); req.pipe(upstream); return; + } + if (req.method === 'GET' && path === '/oidc/.well-known/openid-configuration') return json(200, { + issuer: ISSUER, authorization_endpoint: `${ISSUER}authorize`, token_endpoint: `${ISSUER}token`, jwks_uri: `${ISSUER}jwks`, + response_types_supported: ['code'], subject_types_supported: ['public'], id_token_signing_alg_values_supported: ['RS256'], + token_endpoint_auth_methods_supported: ['client_secret_basic'], code_challenge_methods_supported: ['S256'], + }); + if (req.method === 'GET' && path === '/oidc/jwks') return json(200, { keys: [jwk] }); + if (req.method === 'GET' && path === '/oidc/authorize') { + const q = url.searchParams; + for (const k of new Set(q.keys())) if (q.getAll(k).length !== 1) return json(400, { error: 'invalid_request' }); + if (q.get('client_id') !== CLIENT || q.get('redirect_uri') !== `${ORIGIN}/auth/callback` || q.get('response_type') !== 'code' || + !q.get('scope')?.split(' ').includes('openid') || q.get('code_challenge_method') !== 'S256' || !/^[A-Za-z0-9_-]{43}$/.test(q.get('code_challenge') ?? '') || + !q.get('state') || q.get('state').length > 512 || !q.get('nonce') || q.get('nonce').length > 512 || q.has('login_hint')) return json(400, { error: 'invalid_request' }); + const now = clock(); + for (const [code, grant] of codes) if (grant.expires <= now) codes.delete(code); + if (codes.size >= 100) return json(429, { error: 'too_many_grants' }); + const code = randomBytes(32).toString('base64url'); + codes.set(code, { expires: now + 60000, challenge: q.get('code_challenge'), nonce: q.get('nonce') }); + const callback = new URL(`${ORIGIN}/auth/callback`); callback.searchParams.set('code', code); callback.searchParams.set('state', q.get('state')); + res.writeHead(302, { location: callback.href }); res.end(); return; + } + if (req.method === 'POST' && path === '/oidc/token') { + let body = ''; + req.on('data', c => { body += c; if (body.length > 8192) req.destroy(); }); + req.on('end', () => { + let credentials; + try { credentials = Buffer.from((req.headers.authorization ?? '').replace(/^Basic /, ''), 'base64').toString().split(':').map(decodeURIComponent); } catch { return json(401, { error: 'invalid_client' }); } + if (!req.headers.authorization?.startsWith('Basic ') || credentials.length !== 2 || credentials[0] !== CLIENT || credentials[1] !== secret) return json(401, { error: 'invalid_client' }); + const form = new URLSearchParams(body), code = form.get('code'), grant = codes.get(code); codes.delete(code); + if (!grant || grant.expires <= clock() || form.get('grant_type') !== 'authorization_code' || form.get('redirect_uri') !== `${ORIGIN}/auth/callback` || + createHash('sha256').update(form.get('code_verifier') ?? '').digest('base64url') !== grant.challenge) return json(400, { error: 'invalid_grant' }); + const now = Math.floor(clock() / 1000); + const id_token = jwt({ alg: 'RS256', kid: jwk.kid }, { iss: ISSUER, sub: SUBJECT, aud: CLIENT, name: OWNER, nonce: grant.nonce, iat: now, exp: now + 300 }, { key: privateKey }); + return json(200, { access_token: randomBytes(32).toString('base64url'), token_type: 'Bearer', expires_in: 300, id_token }); + }); return; + } + json(404, { error: 'not_found' }); + }); + await new Promise((resolve, reject) => { server.once('error', reject); server.listen(17443, '127.0.0.1', resolve); }); + return server; +} +export async function main(argv) { + const command = argv.shift(), o = options(argv, command === 'prepare' ? ['--instance','--config','--owner','--runtime','--tls-port'] : ['--runtime']); + if (command === 'prepare') console.log(JSON.stringify(prepare({ instance:o['--instance'], config:o['--config'], owner:o['--owner'], runtime:o['--runtime'], tlsPort:o['--tls-port'] }), null, 2)); + else if (command === 'serve') { const server = await serve(o['--runtime']); for (const signal of ['SIGINT','SIGTERM']) process.once(signal, () => { server.closeAllConnections(); server.close(); }); } + else fail('use prepare or serve'); +} +if (isMain(import.meta.url)) main(process.argv.slice(2)).catch(() => { console.error('isolated identity refused; check named test config, private runtime and free test ports'); process.exitCode = 1; }); diff --git a/packages/cli/live/isolated-identity.test.mjs b/packages/cli/live/isolated-identity.test.mjs new file mode 100644 index 000000000..0647a173b --- /dev/null +++ b/packages/cli/live/isolated-identity.test.mjs @@ -0,0 +1,95 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createHash, verify, createPublicKey } from 'node:crypto'; +import { request } from 'node:https'; +import { createServer } from 'node:http'; +import { readFileSync, statSync, writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { spawn } from 'node:child_process'; +import { once } from 'node:events'; +import { prepare,serve,SUBJECT } from './isolated-identity.mjs'; +import { fixture } from './test-support.mjs'; +import { readSlot } from '../../webui/src/sso.mjs'; +function setup(t) { + const f=fixture(t),runtime=join(f.root,'identity'); + const output=prepare({instance:'ui797',config:f.config,owner:'ui797-owner',runtime,tlsPort:17443}); + return {...f,runtime,output}; +} +function http(runtime,path,options={}) { + return new Promise((resolve,reject)=>{ + const req=request(`https://127.0.0.1:17443${path}`,{ca:readFileSync(join(runtime,'ca.pem')),...options},res=>{ + let body='';res.on('data',c=>body+=c);res.on('end',()=>resolve({status:res.statusCode,headers:res.headers,body})); + });req.on('error',reject);req.end(options.body); + }); +} +test('prepare writes a valid SSO block, private CA/keys and a real slot, never rewrites config',t=>{ + const f=setup(t),before=readFileSync(f.config,'utf8'); + const p=f.output.console.sso.providers[0]; + assert.deepEqual(p.subjects,[{sub:SUBJECT,name:'ui797-owner'}]); + assert.ok(readSlot(join(f.root,'mosaic-ui797'),p,{repo:process.cwd()}).length>30); + for(const n of ['ca.key','tls.key','signing.pem','identity.json','ca.pem']) assert.equal(statSync(join(f.runtime,n)).mode&0o777,0o600); + assert.equal(readFileSync(f.config,'utf8'),before); + assert.throws(()=>prepare({instance:'ui797',config:f.config,owner:'ui797-owner',runtime:f.runtime,tlsPort:17443})); +}); +test('live/default instances, wrong owner, unclaimed root and unsafe config paths refuse before creating runtime',t=>{ + const f=fixture(t); + const defaults={instance:'ui797',config:f.config,owner:'ui797-owner',runtime:join(f.root,'id'),tlsPort:17443}; + for(const instance of ['dev','prod','production'])assert.throws(()=>prepare({...defaults,instance})); + assert.throws(()=>prepare({...defaults,owner:'jason'})); + assert.throws(()=>prepare({...defaults,config:join(f.root,'mosaic-dev/config.json')})); + assert.throws(()=>prepare({...defaults,tlsPort:7332})); + assert.throws(()=>prepare({...defaults,config:join(process.cwd(),'packages/cli/live/..test/mosaic-ui797/config.json')}),/outside source/); + writeFileSync(join(f.dataRoot,'instance.json'),'{"instanceVersion":1,"name":"dev"}'); + assert.throws(()=>prepare(defaults)); +}); +test('real HTTPS authorization redirects, PKCE and nonce bind a single-use bounded RS256 token',async t=>{ + const f=setup(t),server=await serve(f.runtime);t.after(()=>{server.closeAllConnections();server.close();}); + const discovery=await http(f.runtime,'/oidc/.well-known/openid-configuration');assert.equal(discovery.status,200); + const verifier='a'.repeat(43),challenge=createHash('sha256').update(verifier).digest('base64url'); + const q=new URLSearchParams({client_id:'mosaic-ui797',redirect_uri:'https://127.0.0.1:17443/auth/callback',response_type:'code',scope:'openid',state:'test-state',nonce:'test-nonce',code_challenge:challenge,code_challenge_method:'S256'}); + const auth=await http(f.runtime,'/oidc/authorize?'+q);assert.equal(auth.status,302); + const callback=new URL(auth.headers.location);assert.equal(callback.searchParams.get('state'),'test-state'); + const secret=readFileSync(join(f.root,'mosaic-ui797/secrets/mosaic-stack/ui797-local'),'utf8').trim(); + const tokenOptions={method:'POST',headers:{authorization:'Basic '+Buffer.from('mosaic-ui797:'+secret).toString('base64'),'content-type':'application/x-www-form-urlencoded'},body:new URLSearchParams({grant_type:'authorization_code',code:callback.searchParams.get('code'),redirect_uri:'https://127.0.0.1:17443/auth/callback',code_verifier:verifier}).toString()}; + const token=await http(f.runtime,'/oidc/token',tokenOptions);assert.equal(token.status,200); + const [head,payload,sig]=JSON.parse(token.body).id_token.split('.'),claims=JSON.parse(Buffer.from(payload,'base64url')); + assert.equal(claims.sub,SUBJECT);assert.equal(claims.nonce,'test-nonce');assert.equal(claims.exp-claims.iat,300); + const jwks=JSON.parse((await http(f.runtime,'/oidc/jwks')).body); + assert.ok(verify('sha256',Buffer.from(head+'.'+payload),createPublicKey({key:jwks.keys[0],format:'jwk'}),Buffer.from(sig,'base64url'))); + assert.equal((await http(f.runtime,'/oidc/token',tokenOptions)).status,400); + q.set('redirect_uri','https://mosaic.woltje.com/auth/callback');assert.equal((await http(f.runtime,'/oidc/authorize?'+q)).status,400); + q.set('redirect_uri','https://127.0.0.1:17443/auth/callback');q.set('login_hint','jason');assert.equal((await http(f.runtime,'/oidc/authorize?'+q)).status,400);q.delete('login_hint'); + const second=new URL((await http(f.runtime,'/oidc/authorize?'+q)).headers.location); + const wrong={...tokenOptions,body:new URLSearchParams({grant_type:'authorization_code',code:second.searchParams.get('code'),redirect_uri:'https://127.0.0.1:17443/auth/callback',code_verifier:'wrong'}).toString()}; + assert.equal((await http(f.runtime,'/oidc/token',wrong)).status,400); + assert.equal((await http(f.runtime,'/oidc/jwks',{servername:'127.0.0.1',headers:{host:'mosaic.woltje.com'}})).status,403); +}); +test('fixed TLS forwarder preserves normal Console Host, cookies and origin without injected identity',async t=>{ + const f=setup(t);let headers; + const upstream=createServer((req,res)=>{headers=req.headers;res.end('Console upstream');}); + await new Promise((resolve,reject)=>{upstream.once('error',reject);upstream.listen(17440,'127.0.0.1',resolve);});t.after(()=>upstream.close()); + const server=await serve(f.runtime);t.after(()=>{server.closeAllConnections();server.close();}); + const r=await http(f.runtime,'/test',{headers:{origin:'https://127.0.0.1:17443',cookie:'fixture=value'}}); + assert.equal(r.body,'Console upstream');assert.equal(headers.host,'127.0.0.1:17443');assert.equal(headers.origin,'https://127.0.0.1:17443');assert.equal(headers.cookie,'fixture=value');assert.equal(headers['x-auth-subject'],undefined); +}); +test('production Console CLI discovers this OP through NODE_EXTRA_CA_CERTS with ordinary TLS validation',async t=>{ + const f=setup(t),config=JSON.parse(readFileSync(f.config));config.console=f.output.console;writeFileSync(f.config,JSON.stringify(config)); + const server=await serve(f.runtime);t.after(()=>{server.closeAllConnections();server.close();}); + const child=spawn(process.execPath,['packages/webui/src/cli.mjs','serve','--port','17440','--board','http://127.0.0.1:17439','--sso'],{env:{...process.env,MOSAIC_CONFIG:f.config,NODE_EXTRA_CA_CERTS:join(f.runtime,'ca.pem')},stdio:['ignore','pipe','pipe']}); + t.after(async()=>{if(child.exitCode===null){child.kill('SIGTERM');await once(child,'exit');}}); + let output='';child.stdout.on('data',c=>output+=c);child.stderr.on('data',c=>output+=c); + await new Promise((resolve,reject)=>{const timer=setTimeout(()=>reject(new Error('Console startup: '+output)),15000);child.stdout.on('data',()=>{if(output.includes('http://127.0.0.1:17440')){clearTimeout(timer);resolve();}});child.once('exit',()=>{clearTimeout(timer);reject(new Error('Console refused: '+output));});}); + const health=JSON.parse((await http(f.runtime,'/healthz')).body);assert.equal(health.sso['ui797-local'],'ready'); +}); +test('authorization codes expire, and tampering with the single subject refuses service',async t=>{ + const f=setup(t);let now=Date.now(); + const server=await serve(f.runtime,{clock:()=>now});t.after(()=>{server.closeAllConnections();server.close();}); + const verifier='v'.repeat(43),challenge=createHash('sha256').update(verifier).digest('base64url'); + const q=new URLSearchParams({client_id:'mosaic-ui797',redirect_uri:'https://127.0.0.1:17443/auth/callback',response_type:'code',scope:'openid',state:'state',nonce:'nonce',code_challenge:challenge,code_challenge_method:'S256'}); + const code=new URL((await http(f.runtime,'/oidc/authorize?'+q)).headers.location).searchParams.get('code');now+=60001; + const secret=readFileSync(join(f.root,'mosaic-ui797/secrets/mosaic-stack/ui797-local'),'utf8').trim(); + const response=await http(f.runtime,'/oidc/token',{method:'POST',headers:{authorization:'Basic '+Buffer.from('mosaic-ui797:'+secret).toString('base64')},body:new URLSearchParams({grant_type:'authorization_code',code,redirect_uri:'https://127.0.0.1:17443/auth/callback',code_verifier:verifier}).toString()}); + assert.equal(response.status,400); + const stateFile=join(f.runtime,'identity.json'),state=JSON.parse(readFileSync(stateFile));state.console.sso.providers[0].subjects=[{sub:'jason',name:'jason'}];writeFileSync(stateFile,JSON.stringify(state)); + await assert.rejects(serve(f.runtime),/metadata refused/); +}); diff --git a/packages/cli/live/support-common.mjs b/packages/cli/live/support-common.mjs new file mode 100644 index 000000000..b741d4343 --- /dev/null +++ b/packages/cli/live/support-common.mjs @@ -0,0 +1,71 @@ +// Owner-run, local test support. No product capability or processing-state writes. +import { constants, closeSync, fsyncSync, fstatSync, lstatSync, mkdirSync, openSync, readFileSync, realpathSync, writeFileSync } from 'node:fs'; +import { dirname, isAbsolute, join, resolve, relative } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { spawnSync } from 'node:child_process'; +import { homedir } from 'node:os'; +export const ROOT = fileURLToPath(new URL('../../../', import.meta.url)); +export const fail = message => { throw new Error(message); }; +export const inside = (parent, path) => { const r = relative(parent, path); return !r || (r !== '..' && !r.startsWith('../') && !isAbsolute(r)); }; +export function plainPath(path) { + if (!isAbsolute(path ?? '') || resolve(path) !== path) fail('canonical absolute path required'); + let cursor = path; + while (cursor !== dirname(cursor)) { + try { if (lstatSync(cursor).isSymbolicLink()) fail('symbolic link refused'); } + catch (e) { if (e.code !== 'ENOENT') throw e; } + cursor = dirname(cursor); + } + return path; +} +export function privateDir(path, create = false) { + plainPath(path); + if (create) mkdirSync(path, { mode: 0o700 }); + const s = lstatSync(path); + if (!s.isDirectory() || s.uid !== process.getuid() || (s.mode & 0o777) !== 0o700) fail('private owner directory required'); + return path; +} +export function readPrivate(file) { + plainPath(file); + const fd = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK); + try { + const s = fstatSync(fd); + if (!s.isFile() || s.uid !== process.getuid() || (s.mode & 0o777) !== 0o600 || s.size > 16 * 1024 * 1024) fail('private bounded owner file required'); + return readFileSync(fd, 'utf8'); + } finally { closeSync(fd); } +} +export function exclusive(file, value) { + plainPath(file); + const fd = openSync(file, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600); + try { writeFileSync(fd, value); fsyncSync(fd); } finally { closeSync(fd); } +} +export function options(argv, allowed) { + const out = {}; + while (argv.length) { + const key = argv.shift(); + if (!allowed.includes(key) || Object.hasOwn(out, key) || !argv.length) fail('unknown, duplicate or missing option'); + out[key] = argv.shift(); + } + return out; +} +export function testInstance(instance, config) { + if (!/^[a-z][a-z0-9]{1,15}$/.test(instance ?? '') || ['dev','prod','production'].includes(instance)) fail('explicit named test instance required'); + plainPath(config); + const copies = join(process.env.XDG_DATA_HOME || join(homedir(), '.local/share'), 'mosaic-stack/copies'); + if (inside(ROOT, config) || inside(copies, config)) fail('test config and secret slots must be outside source and stack copies'); + if (config.split('/').includes('mosaic-dev') || config.split('/').includes('.mosaic-dev') || dirname(config).split('/').at(-1) !== `mosaic-${instance}`) fail('matching named test config required; mosaic-dev refused'); + if (process.env.NODE_TLS_REJECT_UNAUTHORIZED === '0') fail('TLS verification must remain enabled'); + const run = spawnSync(process.execPath, [join(ROOT, 'scripts/mosaic-config.mjs'), 'validate'], { env: { ...process.env, MOSAIC_CONFIG: config }, encoding: 'utf8' }); + if (run.status !== 0) fail('test system configuration validation refused'); + const system = JSON.parse(run.stdout); + plainPath(system.dataRoot); + if (inside(ROOT, system.dataRoot) || inside(copies, system.dataRoot)) fail('test data root must be outside source and stack copies'); + if (system.environment !== 'development' || system.dataRoot.split('/').some(p => ['.mosaic-dev','mosaic-dev'].includes(p))) fail('isolated development data root required'); + const marker = JSON.parse(readFileSync(join(system.dataRoot, 'instance.json'), 'utf8')); + if (marker.instanceVersion !== 1 || marker.name !== instance) fail('test instance marker mismatch'); + return { system, config: realpathSync(config) }; +} +export function outsideRuntime(path, system, config) { + plainPath(path); + if (inside(ROOT, path) || inside(system.dataRoot, path) || inside(dirname(config), path)) fail('runtime must be outside repository, config and model data'); +} +export function isMain(meta) { return process.argv[1] && fileURLToPath(meta) === resolve(process.argv[1]); } diff --git a/packages/cli/live/test-support.mjs b/packages/cli/live/test-support.mjs new file mode 100644 index 000000000..5b2e20b2b --- /dev/null +++ b/packages/cli/live/test-support.mjs @@ -0,0 +1,15 @@ +// Focused test fixtures only; runtime CLIs never import this file. +import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +export function fixture(t) { + if (!process.env.TMPDIR?.startsWith('/mnt/storage/scratch/tmp/') || process.env.TMPDIR.includes('_')) throw new Error('set TMPDIR with mktemp -d /mnt/storage/scratch/tmp/XXXXXX'); + const root=mkdtempSync(join(tmpdir(),'ui797-')); + t.after(()=>rmSync(root,{recursive:true,force:true})); + const configDir=join(root,'mosaic-ui797'),dataRoot=join(root,'data'); + mkdirSync(configDir,{mode:0o700});mkdirSync(dataRoot,{mode:0o700}); + const config=join(configDir,'config.json'); + writeFileSync(config,JSON.stringify({configVersion:1,environment:'development',dataRoot,execution:{backend:'docker',provider:'zai',model:'glm-5.3-flash'}})); + writeFileSync(join(dataRoot,'instance.json'),JSON.stringify({instanceVersion:1,name:'ui797'})); + return {root,config,dataRoot}; +} diff --git a/packages/cli/live/ui-browser.mjs b/packages/cli/live/ui-browser.mjs new file mode 100644 index 000000000..d3d421d17 --- /dev/null +++ b/packages/cli/live/ui-browser.mjs @@ -0,0 +1,279 @@ +#!/usr/bin/env node +// Persistent CDP pipe client, based on webui/tests/browser.mjs. No injected app script. +import { spawn, fork, spawnSync } from 'node:child_process'; +import { createServer, createConnection } from 'node:net'; +import { once } from 'node:events'; +import { createHash, X509Certificate } from 'node:crypto'; +import { appendFileSync, chmodSync, existsSync, readdirSync, readFileSync, unlinkSync } from 'node:fs'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { homedir } from 'node:os'; +import { exclusive, fail, isMain, options, plainPath, privateDir, readPrivate } from './support-common.mjs'; +const HERE = fileURLToPath(import.meta.url); +const INTERACTIVE = new Set(['button','link','textbox','searchbox','combobox','checkbox','radio','switch','tab','menuitem','option','spinbutton','slider']); +const delay = ms => new Promise(r => setTimeout(r, ms)); +export function testURL(value, origin = null) { + const u = new URL(value, origin ?? undefined); + if (u.hostname !== '127.0.0.1' || !['17440','17443'].includes(u.port) || !['http:','https:'].includes(u.protocol) || u.username || u.password || + (u.port === '17443' && u.protocol !== 'https:') || (u.port === '17440' && u.protocol !== 'http:')) fail('only loopback test Console ports 17440/17443 allowed'); + if (origin && u.origin !== origin) fail('navigation must stay on the selected test origin'); + return u; +} +async function chromium(evidence, ca) { + const cert = new X509Certificate(readPrivate(ca)); + if (!cert.ca || Date.parse(cert.validTo) <= Date.now() || Date.parse(cert.validFrom) > Date.now()) fail('valid private test CA required'); + const xdg = join(evidence, 'xdg'); privateDir(xdg, true); + const pki = join(xdg, 'pki'); privateDir(pki, true); + const nss = join(pki, 'nssdb'); privateDir(nss, true); + for (const args of [['-N','--empty-password','-d',`sql:${nss}`], ['-A','-d',`sql:${nss}`,'-n','ui797-private-test-ca','-t','C,,','-i',ca]]) { + const r = spawnSync('certutil', args, { encoding:'utf8' }); + if (r.status !== 0) fail('private Chromium CA trust setup failed (certutil required)'); + } + for (const name of readdirSync(nss)) chmodSync(join(nss,name),0o600); + const profile = join(evidence, 'profile'); privateDir(profile, true); + // NSS prefers ~/.pki/nssdb when it exists, even with XDG_DATA_HOME set. Overlay it + // in a private mount namespace, never mutate the owner's trust store or change HOME. + // On newer clean hosts, the process-local XDG database alone is sufficient. + const legacyNss = join(homedir(), '.pki/nssdb'); + const legacy = existsSync(legacyNss); + if (legacy) plainPath(legacyNss); + const executable = process.env.CHROMIUM || '/usr/bin/chromium'; + const chromeArgs = ['--headless','--no-first-run','--no-default-browser-check','--disable-dev-shm-usage', + '--disable-background-networking','--disable-component-update','--disable-sync','--no-proxy-server','--remote-debugging-pipe', + `--user-data-dir=${profile}`]; + const child = spawn(legacy ? '/usr/bin/bwrap' : executable, legacy ? ['--die-with-parent','--bind','/','/','--dev-bind','/dev','/dev','--proc','/proc','--bind',nss,legacyNss,'--',executable,...chromeArgs] : chromeArgs, + { env:{...process.env,XDG_DATA_HOME:xdg}, stdio: ['ignore','ignore','ignore','pipe','pipe'] }); + let id = 0, buffer = ''; + const pending = new Map(), events = new Map(); + const rejectAll = () => { for (const p of pending.values()) { clearTimeout(p.timer); p.reject(new Error('Chromium exited')); } pending.clear(); }; + child.once('error', rejectAll); child.once('exit', rejectAll); + child.stdio[3].on('error', rejectAll); child.stdio[4].on('error', rejectAll); + child.stdio[4].on('data', chunk => { + buffer += chunk.toString(); let end; + while ((end = buffer.indexOf('\0')) !== -1) { + const msg = JSON.parse(buffer.slice(0,end)); buffer = buffer.slice(end+1); + if (msg.method) for (const f of events.get(msg.method) ?? []) f(msg.params, msg.sessionId); + const p = pending.get(msg.id); + if (p) { pending.delete(msg.id); clearTimeout(p.timer); msg.error ? p.reject(new Error('CDP refused ' + p.method)) : p.resolve(msg.result); } + } + }); + const send = (method, params = {}, sessionId) => new Promise((resolve, reject) => { + const key = ++id, timer = setTimeout(() => { pending.delete(key); reject(new Error('CDP timeout: ' + method)); }, 15000); + pending.set(key, { resolve, reject, timer, method }); + child.stdio[3].write(JSON.stringify({ id:key, method, params, ...(sessionId ? { sessionId } : {}) }) + '\0'); + }); + const on = (method, f) => { if (!events.has(method)) events.set(method,new Set()); events.get(method).add(f); }; + const { targetId } = await send('Target.createTarget', { url:'about:blank' }); + const { sessionId } = await send('Target.attachToTarget', { targetId, flatten:true }); + const call = (method, params) => send(method, params, sessionId); + return { call, send, on, child, async close() { + await send('Browser.close').catch(() => {}); + if (child.exitCode === null) await once(child, 'exit'); + } }; +} +export async function daemon(evidence) { + privateDir(evidence); + const state = JSON.parse(readPrivate(join(evidence, 'browser.json'))), origin = testURL(state.origin).origin; + const b = await chromium(evidence, state.ca); + try { + let sequence = 0, generation = 0, refs = new Map(), blocked = false, detail = {}; + const completed = new Map(), pendingDownloads = new Set(); + const downloadsDir = join(evidence,'downloads'); privateDir(downloadsDir, true); + b.on('Browser.downloadWillBegin', e => { pendingDownloads.add(e.guid); }); + b.on('Browser.downloadProgress', e => { if (e.state === 'completed') { completed.set(e.guid, true); pendingDownloads.delete(e.guid); } }); + b.on('Page.frameNavigated', e => { if (!e.frame.parentId) { generation++; refs.clear(); } }); + const refused = async id => { + blocked = true; + await b.call('Fetch.failRequest', { requestId:id, errorReason:'BlockedByClient' }).catch(() => {}); + }; + // Includes subresources, redirects and popups. No external live Console can be contacted. + b.on('Fetch.requestPaused', e => { + let allowed = false; + try { allowed = testURL(e.request.url).origin === origin; } catch {} + (allowed ? b.call('Fetch.continueRequest', { requestId:e.requestId }) : refused(e.requestId)).catch(() => { blocked = true; }); + }); + // Deny new targets (including window.open) before they are allowed to run. + b.on('Target.attachedToTarget', e => { + if (e.waitingForDebugger) b.send('Target.closeTarget', { targetId:e.targetInfo.targetId }).catch(() => {}); + }); + await b.send('Target.setAutoAttach', { autoAttach:true, waitForDebuggerOnStart:true, flatten:true }); + await b.call('Page.enable'); await b.call('Accessibility.enable'); await b.call('DOM.enable'); + await b.call('Network.enable'); await b.call('Network.setBlockedURLs', { urls:['ws://*','wss://*'] }); + await b.call('Fetch.enable', { patterns:[{ urlPattern:'*', requestStage:'Request' }] }); + await b.send('Browser.setDownloadBehavior', { behavior:'allowAndName', downloadPath:downloadsDir, eventsEnabled:true }); + await b.call('Emulation.setDeviceMetricsOverride', { width:1280, height:1000, deviceScaleFactor:1, mobile:false }); + const shot = async name => { const { data } = await b.call('Page.captureScreenshot', { format:'png' }); exclusive(join(evidence,name), Buffer.from(data,'base64')); return name; }; + async function snapshot() { + const { nodes } = await b.call('Accessibility.getFullAXTree'); + const text = [], elements = []; + const next = new Map(); + for (const n of nodes) { + if (n.ignored) continue; + const role = n.role?.value, name = n.name?.value ?? ''; + if (role === 'StaticText' && name) text.push(name); + if (INTERACTIVE.has(role) && n.backendDOMNodeId) { + const ref = `r${generation}-${n.backendDOMNodeId}`; + next.set(ref,n.backendDOMNodeId); + // Values (including secret input) never enter snapshots or receipts. + elements.push({ ref, role, name, disabled:n.properties?.some(p => p.name === 'disabled' && p.value?.value === true) ?? false }); + } + } + refs = next; + return { text:text.join('\n'), elements }; + } + const currentURL = async () => { + const { frameTree } = await b.call('Page.getFrameTree'); + if (frameTree.frame.url !== 'about:blank') testURL(frameTree.frame.url, origin); + }; + async function focus(ref) { + const backendNodeId = refs.get(ref); if (!backendNodeId) fail('unknown or stale ref; snapshot again'); + await b.call('DOM.scrollIntoViewIfNeeded', { backendNodeId }); + const { quads } = await b.call('DOM.getContentQuads', { backendNodeId }); + const q = quads?.[0]; if (!q || Math.abs((q[2]-q[0])*(q[5]-q[1])) < 1) fail('element has no visible target'); + const x = (q[0]+q[2]+q[4]+q[6])/4, y = (q[1]+q[3]+q[5]+q[7])/4; + await b.call('Input.dispatchMouseEvent', { type:'mouseMoved',x,y }); + await b.call('Input.dispatchMouseEvent', { type:'mousePressed',x,y,button:'left',clickCount:1 }); + await b.call('Input.dispatchMouseEvent', { type:'mouseReleased',x,y,button:'left',clickCount:1 }); + } + async function press(key) { + const parts = key.split('+'), base = parts.pop(); let modifiers = 0; + for (const part of parts) { + const mask = { Alt:1, Ctrl:2, Control:2, Meta:4, Shift:8 }[part]; + if (!mask || (modifiers & mask)) fail('unsupported key modifier'); + modifiers |= mask; + } + let spec = { Enter:['Enter',13,'\r'], Tab:['Tab',9,''], Escape:['Escape',27,''], Backspace:['Backspace',8,''], Delete:['Delete',46,''], Home:['Home',36,''], End:['End',35,''], PageUp:['PageUp',33,''], PageDown:['PageDown',34,''], ArrowDown:['ArrowDown',40,''], ArrowUp:['ArrowUp',38,''], ArrowLeft:['ArrowLeft',37,''], ArrowRight:['ArrowRight',39,''], Space:['Space',32,' '] }[base]; + if (!spec && /^[a-zA-Z0-9]$/.test(base)) spec = [/[a-zA-Z]/.test(base) ? `Key${base.toUpperCase()}` : `Digit${base}`,base.toUpperCase().charCodeAt(0),base]; + if (!spec) fail('unsupported key'); + detail.key=key; + const event = { key:base === 'Space' ? ' ' : base,code:spec[0],windowsVirtualKeyCode:spec[1],modifiers }; + await b.call('Input.dispatchKeyEvent', { ...event,type:'keyDown',text:modifiers ? '' : spec[2] }); + await b.call('Input.dispatchKeyEvent', { ...event,type:'keyUp' }); + } + async function navigate(path) { + const url = testURL(path,origin); + // Authentication query strings are never accepted as CLI inputs or retained. + if (url.pathname === '/auth/callback' || url.pathname.startsWith('/oidc/')) fail('navigate via the Console sign-in control'); + const r = await b.call('Page.navigate', { url:url.href }); + if (r.errorText) fail('navigation failed'); + for (let n=0;n<100;n++) { + const { root } = await b.call('DOM.getDocument'); + if (root.childNodeCount > 0) { await delay(100); return; } + await delay(50); + } + fail('navigation timed out'); + } + async function perform(command,args) { + switch (command) { + case 'start': await navigate('/'); return snapshot(); + case 'goto': if (args.length !== 1) fail('goto PATH'); await navigate(args[0]); return snapshot(); + case 'snapshot': if (args.length) fail('snapshot takes no arguments'); return snapshot(); + case 'click': { + let ref; + if (args.length === 2 && args[0] === '--text') { + const s = await snapshot(), found = s.elements.filter(e => e.name === args[1] && !e.disabled); + if (found.length !== 1) fail('text must name exactly one enabled interactive element'); ref = found[0].ref; + } else if (args.length === 1) ref = args[0]; else fail('click REF or --text TEXT'); + if (/^r[0-9]+-[0-9]+$/.test(ref)) detail.ref = ref; await focus(ref); await delay(150); return snapshot(); + } + case 'type': if (args.length !== 2) fail('type REF TEXT'); if (/^r[0-9]+-[0-9]+$/.test(args[0])) detail.ref=args[0]; await focus(args[0]); await b.call('Input.insertText', { text:args[1] }); return snapshot(); + case 'press': if (args.length !== 1) fail('press KEY'); await press(args[0]); await delay(100); return snapshot(); + case 'wait': { + const o = options([...args],['--text','--timeout']); const timeout = Number(o['--timeout'] ?? 10); + if (!o['--text'] || !Number.isFinite(timeout) || timeout <= 0 || timeout > 120) fail('wait --text TEXT --timeout 0..120 seconds'); + const end = Date.now()+timeout*1000; + do { const s = await snapshot(); if (s.text.includes(o['--text']) || s.elements.some(e => e.name.includes(o['--text']))) return s; await delay(100); } while (Date.now() { const file = join(downloadsDir,name), bytes = readFileSync(file); chmodSync(file,0o600); return { file, bytes:bytes.length, sha256:createHash('sha256').update(bytes).digest('hex') }; }); + } + case 'stop': return { stopped:true }; + default: fail('unknown browser command'); + } + } + async function action(command,args) { + detail = {}; + const n = String(++sequence).padStart(5,'0'), before = await shot(`${n}-before.png`); + let result, error, after; + try { if (command !== 'stop') await currentURL(); result = await perform(command,args); if (command !== 'stop') { await currentURL(); if (blocked) fail('non-test network request blocked; stop session'); } } + catch (e) { error = e.message; } + finally { + after = await shot(`${n}-after.png`).catch(() => null); + // No text input, URLs, auth codes, tokens, cookie/header/body values in the journal. + appendFileSync(join(evidence,'actions.jsonl'), JSON.stringify({ sequence,at:new Date().toISOString(),command,...detail,before,after,ok:!error,error:error ?? null })+'\n', { mode:0o600 }); + } + if (error) throw new Error(error); + return result; + } + let tail = Promise.resolve(), stopping = false; + const socket = join(evidence,'browser.sock'); + if (Buffer.byteLength(socket)>100) fail('evidence path too long for Unix socket'); + const server = createServer(peer => { + peer.on('error', () => {}); + peer.setTimeout(150000, () => peer.destroy()); let data = ''; + peer.on('data', chunk => { + data += chunk; if (data.length > 65536) return peer.destroy(); + if (!data.includes('\n')) return; + peer.removeAllListeners('data'); + tail = tail.then(async () => { + let request; + try { + request = JSON.parse(data); if (!Array.isArray(request.args) || request.args.some(a => typeof a !== 'string') || stopping) fail('bad browser command'); + const value = await action(request.command,request.args); + if (request.command === 'stop') { stopping = true; server.close(); await b.close(); if (existsSync(socket)) unlinkSync(socket); } + peer.end(JSON.stringify({ ok:true,value })+'\n'); + } catch (e) { peer.end(JSON.stringify({ ok:false,error:e.message })+'\n'); } + }); + }); + }); + await new Promise((resolve,reject) => { server.once('error',reject); server.listen(socket,resolve); }); chmodSync(socket,0o600); + const cleanup = async () => { server.close(); await b.close(); if (existsSync(socket)) unlinkSync(socket); }; + for (const signal of ['SIGTERM','SIGINT']) process.once(signal, () => cleanup().finally(() => process.exit(0))); + return { server, cleanup }; + } catch (e) { await b.close().catch(() => {}); throw e; } +} +export function callBrowser(evidence, command, args=[]) { + privateDir(evidence); readPrivate(join(evidence,'browser.json')); + return new Promise((resolve,reject) => { + const peer = createConnection(join(evidence,'browser.sock')); let data = ''; + peer.on('error', () => {}); + peer.setTimeout(150000, () => peer.destroy(new Error('browser action timed out; inspect receipts before retry'))); + peer.on('error',reject); peer.on('connect', () => peer.write(JSON.stringify({ command,args })+'\n')); + peer.on('data',chunk => { data += chunk; }); peer.on('end', () => { try { const r = JSON.parse(data); r.ok ? resolve(r.value) : reject(new Error(r.error)); } catch(e) { reject(e); } }); + }); +} +export async function start({ evidence, origin, ca }) { + if (testURL(origin).origin !== origin) fail('origin must have no path, query or fragment'); + plainPath(ca); readPrivate(ca); plainPath(evidence); privateDir(evidence,true); + exclusive(join(evidence,'browser.json'),JSON.stringify({ version:1,origin,ca })+'\n'); + const child = fork(HERE,['--daemon',evidence],{ detached:true,stdio:['ignore','ignore','ignore','ipc'] }); + try { + await new Promise((resolve,reject) => { + const timer = setTimeout(() => { child.kill('SIGTERM'); reject(new Error('browser daemon startup timed out')); },20000); + child.once('error',reject); child.once('exit', () => { clearTimeout(timer); reject(new Error('browser daemon failed to start')); }); + child.once('message',m => { clearTimeout(timer); m.ok ? resolve() : reject(new Error(m.error)); }); + }); + } catch (e) { child.kill('SIGTERM'); if (child.connected) child.disconnect(); child.unref(); throw e; } + child.disconnect(); child.unref(); + return callBrowser(evidence,'start'); +} +export async function main(argv, env=process.env) { + const command = argv.shift(); + if (command === '--daemon') { await daemon(argv[0]); process.send?.({ok:true}); return; } + if (command === 'start') { + const o = options(argv,['--evidence','--origin','--ca']); return start({evidence:o['--evidence'],origin:o['--origin'],ca:o['--ca']}); + } + let evidence = env.UI797_EVIDENCE; + const index = argv.indexOf('--evidence'); + if (index >= 0) { evidence = argv[index+1]; argv.splice(index,2); } + return callBrowser(evidence,command,argv); +} +if (isMain(import.meta.url)) main(process.argv.slice(2)).then(r => { if (r !== undefined) console.log(JSON.stringify(r,null,2)); }).catch(e => { if (process.send) process.send({ok:false,error:'browser daemon startup refused'}); else console.error(e.message); process.exitCode=1; }); diff --git a/packages/cli/live/ui-browser.test.mjs b/packages/cli/live/ui-browser.test.mjs new file mode 100644 index 000000000..cd37efecf --- /dev/null +++ b/packages/cli/live/ui-browser.test.mjs @@ -0,0 +1,91 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createServer } from 'node:http'; +import { readFileSync, readdirSync, existsSync } from 'node:fs'; +import { join } from 'node:path'; +import { spawn } from 'node:child_process'; +import { start,callBrowser,testURL } from './ui-browser.mjs'; +import { prepare,serve } from './isolated-identity.mjs'; +import { fixture } from './test-support.mjs'; +const pause=ms=>new Promise(r=>setTimeout(r,ms)); +test('browser URL guard refuses live, remote, credentials, non-test ports and origin changes',()=>{ + for(const u of ['http://127.0.0.1:7332','https://mosaic.woltje.com','http://10.1.1.53:17440','http://localhost:17440','http://u:p@127.0.0.1:17440','https://127.0.0.1:17440','http://127.0.0.1:17443'])assert.throws(()=>testURL(u)); + assert.equal(testURL('/chat','http://127.0.0.1:17440').pathname,'/chat'); + assert.throws(()=>testURL('https://127.0.0.1:17443','http://127.0.0.1:17440')); +}); +async function setup(t,html) { + const f=fixture(t),identity=join(f.root,'id'); + prepare({instance:'ui797',config:f.config,owner:'ui797-owner',runtime:identity,tlsPort:17443}); + const hits=[]; + const server=createServer((req,res)=>{ + hits.push(req.url); + if(req.url==='/download'){res.writeHead(200,{'content-disposition':'attachment; filename="product.html"','content-type':'text/html'});res.end('

offline artifact

');return;} + if(req.url==='/redirect'){res.writeHead(302,{location:'http://127.0.0.1:7332/live'});res.end();return;} + res.setHeader('content-type','text/html');res.end(html); + }); + await new Promise((resolve,reject)=>{server.once('error',reject);server.listen(17440,'127.0.0.1',resolve);});t.after(()=>server.close()); + const evidence=join(f.root,'browser'); + t.after(async()=>{if(existsSync(join(evidence,'browser.sock')))await callBrowser(evidence,'stop').catch(()=>{});}); + return {...f,identity,evidence,hits}; +} +test('real Chromium keeps a session across CLI processes; mouse/keyboard refs, screenshots and actual downloads',async t=>{ + const f=await setup(t,`fixture

Browser fixture

Download reviewed HTML`); + await start({evidence:f.evidence,origin:'http://127.0.0.1:17440',ca:join(f.identity,'ca.pem')}); + const snapshot=await callBrowser(f.evidence,'snapshot');assert.match(snapshot.text,/Browser fixture/); + const input=snapshot.elements.find(e=>e.name==='Request');assert.ok(input); + const again=await callBrowser(f.evidence,'snapshot');assert.equal(again.elements.find(e=>e.name==='Request').ref,input.ref); + await callBrowser(f.evidence,'type',[input.ref,'discard this']); + await callBrowser(f.evidence,'press',['Ctrl+A']); + await callBrowser(f.evidence,'press',['Backspace']); + await callBrowser(f.evidence,'type',[input.ref,'Pomodoro from keyboard']); + await callBrowser(f.evidence,'click',['--text','Send']); + await callBrowser(f.evidence,'wait',['--text','Pomodoro from keyboard','--timeout','2']); + const cli=spawn(process.execPath,['packages/cli/live/ui-browser.mjs','snapshot','--evidence',f.evidence],{stdio:['ignore','pipe','pipe']});let out='',err='';cli.stdout.on('data',c=>out+=c);cli.stderr.on('data',c=>err+=c); + const code=await new Promise(r=>cli.once('exit',r));assert.equal(code,0,err);assert.match(out,/Pomodoro from keyboard/); + await callBrowser(f.evidence,'screenshot',['fixture']); + assert.ok(readFileSync(join(f.evidence,'fixture.png')).length>1000); + await callBrowser(f.evidence,'click',['--text','Download reviewed HTML']); + await pause(300); + const downloads=await callBrowser(f.evidence,'downloads');assert.equal(downloads.length,1);assert.equal(downloads[0].sha256.length,64);assert.equal(readFileSync(downloads[0].file,'utf8'),'

offline artifact

'); + await assert.rejects(callBrowser(f.evidence,'goto',['http://127.0.0.1:7332']),/only loopback/); + await assert.rejects(callBrowser(f.evidence,'click',['bad-ref']),/stale ref/); + await assert.rejects(callBrowser(f.evidence,'wait',['--text','missing','--timeout','0.2']),/timed out/); + await callBrowser(f.evidence,'press',['Tab']);await callBrowser(f.evidence,'stop'); + const receipts=readFileSync(join(f.evidence,'actions.jsonl'),'utf8').trim().split('\n').map(JSON.parse); + assert.equal(receipts.length,18);assert.ok(receipts.some(r=>r.ok===false)); + for(const r of receipts){assert.ok(existsSync(join(f.evidence,r.before)));assert.ok(existsSync(join(f.evidence,r.after)));} + assert.equal(receipts.map(r=>r.sequence).join(','),Array.from({length:18},(_,i)=>i+1).join(',')); + assert.ok(!JSON.stringify(receipts).includes('Pomodoro from keyboard')); + assert.ok(readdirSync(join(f.evidence,'profile')).length>0); +}); +test('redirects to the live port are blocked before a request reaches it',async t=>{ + const f=await setup(t,'

safe

Escape'); + await start({evidence:f.evidence,origin:'http://127.0.0.1:17440',ca:join(f.identity,'ca.pem')}); + await assert.rejects(callBrowser(f.evidence,'click',['--text','Escape']),/blocked|only loopback/); + assert.ok(f.hits.includes('/redirect')); + assert.ok(readFileSync(join(f.evidence,'actions.jsonl'),'utf8').includes('false')); +}); +test('private CA browser performs the actual authorization redirect into production Console SSO',async t=>{ + const {writeFileSync}=await import('node:fs'); + const f=fixture(t),identity=join(f.root,'id'),evidence=join(f.root,'browser'); + const output=prepare({instance:'ui797',config:f.config,owner:'ui797-owner',runtime:identity,tlsPort:17443}); + const config=JSON.parse(readFileSync(f.config));config.console=output.console;writeFileSync(f.config,JSON.stringify(config)); + const idp=await serve(identity);t.after(()=>{idp.closeAllConnections();idp.close();}); + const child=spawn(process.execPath,['packages/webui/src/cli.mjs','serve','--port','17440','--board','http://127.0.0.1:17439','--sso'],{env:{...process.env,MOSAIC_CONFIG:f.config,NODE_EXTRA_CA_CERTS:join(identity,'ca.pem')},stdio:['ignore','pipe','pipe']});let text='';child.stdout.on('data',c=>text+=c);child.stderr.on('data',c=>text+=c); + t.after(async()=>{if(existsSync(join(evidence,'browser.sock')))await callBrowser(evidence,'stop').catch(()=>{});if(child.exitCode===null){child.kill('SIGTERM');await new Promise(r=>child.once('exit',r));}}); + await new Promise((resolve,reject)=>{const timer=setTimeout(()=>reject(new Error(text)),15000);child.stdout.on('data',()=>{if(text.includes('http://127.0.0.1:17440')){clearTimeout(timer);resolve();}});child.once('exit',()=>{clearTimeout(timer);reject(new Error(text));});}); + const initial=await start({evidence,origin:'https://127.0.0.1:17443',ca:join(identity,'ca.pem')}); + // The sole provider signs in immediately through redirects during start. + const signedIn=await callBrowser(evidence,'wait',['--text','Sign out','--timeout','10']); + assert.ok(signedIn.elements.some(e=>e.name==='Sign out')); + assert.ok(!text.includes('sso refused')); + assert.ok(!readFileSync(join(evidence,'actions.jsonl'),'utf8').includes('code=')); +}); +test('a different private CA is refused by Chromium with TLS verification enabled',async t=>{ + const f=fixture(t),g=fixture(t),id=join(f.root,'id'),other=join(g.root,'id'),evidence=join(f.root,'browser'); + for(const [config,runtime] of [[f.config,id],[g.config,other]])prepare({instance:'ui797',config,owner:'ui797-owner',runtime,tlsPort:17443}); + const server=await serve(other);t.after(()=>{server.closeAllConnections();server.close();}); + t.after(async()=>{if(existsSync(join(evidence,'browser.sock')))await callBrowser(evidence,'stop');}); + await assert.rejects(start({evidence,origin:'https://127.0.0.1:17443',ca:join(id,'ca.pem')}),/navigation failed/); + assert.ok(readFileSync(join(evidence,'actions.jsonl'),'utf8').includes('false')); +});