docs(mos): format Option 2 qualification report

docs/SITEMAP.md

@@ -41,3 +41,10 @@
 - [Retention and deprecation evidence](tess/M5-MIGRATION-RETENTION-DEPRECATION.md)
 - [Verification matrix](tess/VERIFICATION-MATRIX.md)
 - [Documentation checklist](tess/M5-003-DOCUMENTATION-CHECKLIST.md)
+- [Independent Option 2 runtime-portability qualification (2026-07-14)](tess/qualification/2026-07-14-option2-runtime-portability.md)
+
+## Runtime-neutral Mos portability
+
+- [Optional AI egress gateway ADR](architecture/ADR-MOS-EGRESS-GATEWAYS.md) — placement and gates for LiteLLM, Bifrost, and purpose-built translation proxies.
+- [Runtime-neutral Mos identity and failover mission](https://git.mosaicstack.dev/mosaicstack/stack/issues/754)
+- [Logical identity and connector lease/fencing implementation](https://git.mosaicstack.dev/mosaicstack/stack/issues/755)

docs/architecture/ADR-MOS-EGRESS-GATEWAYS.md

@@ -0,0 +1,151 @@
+# ADR: Optional AI egress gateways for runtime-neutral Mos
+
+**Status:** Proposed for controlled prototypes; not approved as Mosaic core
+
+**Date:** 2026-07-14
+
+**Issues:** #754, #755
+
+**Decision owner:** Mosaic Gateway / provider-adapter architecture
+
+## Context
+
+The emergency Mos continuity path kept Claude Code as the harness and translated Anthropic Messages traffic to Codex OAuth through a small localhost proxy. That preserved the existing Claude Discord plugin and transcript, but exposed two architectural facts:
+
+1. Harness identity, channel entitlement, provider credentials, and inference transport are separate concerns.
+2. A generic AI gateway can improve provider routing, budgets, and observability, but must not become Mosaic's identity, authorization, tenant, or orchestration boundary.
+
+The Tess qualification report also found that current provider rebinding is not identity-continuous failover. Mosaic still needs a logical agent identity, durable connector lease/fencing, canonical handoff/checkpoint, exactly-once receipts, concrete harness adapters, and cross-harness rollback E2E.
+
+## Decision
+
+Mosaic MAY support LiteLLM, Bifrost, the purpose-built Claude/Codex proxy, or future gateways as optional egress implementations behind `IProviderAdapter` / `AgentRuntimeProvider`.
+
+Mosaic Gateway remains authoritative for:
+
+- authenticated actor and tenant identity;
+- logical agent identity and connector binding;
+- authorization, approval, and policy;
+- lease epoch and stale-holder fencing;
+- audit correlation and redaction;
+- canonical handoff/checkpoint state;
+- idempotency and side-effect receipts.
+
+An egress gateway MUST NOT:
+
+- receive channel ingress directly;
+- authorize tools or connector ownership;
+- define Mosaic tenant or agent identity;
+- persist raw Mosaic handoffs or channel credentials;
+- bypass adapter capability negotiation;
+- silently fail over when policy, lease, or provider health is uncertain.
+
+Allowed topology:
+
+```text
+Discord / Matrix / CLI / web
+        ↓
+Mosaic Gateway: identity, authz, lease/fence, approvals, audit
+        ↓
+IProviderAdapter / AgentRuntimeProvider
+        ↓
+optional egress gateway
+        ↓
+upstream provider or subscription-backed OAuth session
+```
+
+## Candidate assessment
+
+### Purpose-built `raine/claude-code-proxy`
+
+**Disposition:** Approved only for the verified emergency localhost bridge.
+
+Strengths:
+
+- explicit Codex device OAuth flow;
+- small operational surface;
+- Anthropic Messages translation suitable for Claude Code;
+- model and reasoning-effort enforcement;
+- straightforward loopback systemd supervision and rollback.
+
+Constraints:
+
+- not a Mosaic multi-tenant control plane;
+- Claude built-in channels still depend on Claude subscription entitlement and feature lookup;
+- model aliases can obscure the upstream model unless proxy policy/logs are treated as evidence;
+- no replacement for connector leasing, canonical handoff, or exactly-once effects.
+
+### LiteLLM
+
+**Disposition:** Candidate for a formal adapter-only prototype and terms/security review.
+
+Current documentation states that ChatGPT subscription access is available through an OAuth device-code flow. LiteLLM also provides broad provider routing, virtual keys, budgets, observability, and OpenAI/Anthropic-compatible surfaces.
+
+Required prototype gates:
+
+- verify the exact ChatGPT subscription OAuth flow and supported models against current provider terms;
+- document token location, encryption, revocation, refresh, scope, and incident response;
+- prove tenant isolation and prevent virtual keys from becoming Mosaic principals;
+- verify streaming, tool calls, reasoning controls, cancellation, and idempotency metadata;
+- fail closed instead of selecting an unhealthy provider merely to return a result;
+- demonstrate that Mosaic audit correlation survives gateway retries/failover;
+- keep channel ingress and connector credentials outside LiteLLM.
+
+Source references:
+
+- [LiteLLM ChatGPT subscription provider](https://docs.litellm.ai/docs/providers/chatgpt)
+- [LiteLLM providers](https://docs.litellm.ai/docs/providers)
+
+### Bifrost
+
+**Disposition:** Candidate for governance/routing research; subscription OAuth compatibility unverified.
+
+Useful concepts include virtual keys, budgets, rate limits, weighted load balancing, and automatic provider failover. Those features may inform Mosaic egress policy, but Bifrost virtual keys are downstream credentials—not Mosaic actors or tenants.
+
+Required prototype gates:
+
+- verify Codex/ChatGPT subscription OAuth rather than assuming API-key compatibility;
+- map budgets and virtual keys to server-derived Mosaic tenants without duplicating authority;
+- prove failover does not violate connector lease, approval, or exactly-once semantics;
+- ensure request/response logs are redacted before persistence;
+- disable or constrain automatic failover when policy or side-effect state is ambiguous.
+
+Source references:
+
+- [Bifrost overview](https://docs.getbifrost.ai/overview)
+- [Bifrost repository](https://github.com/maximhq/bifrost)
+
+### `teremterem/claude-code-gpt-5-codex`
+
+**Disposition:** Not selected as the emergency implementation; useful as a historical LiteLLM recipe.
+
+The reviewed repository uses `OPENAI_API_KEY`, tells previously authenticated Claude users to log out, and documents a Claude Web Search schema incompatibility. Logging Claude out conflicts with the channel-entitlement requirement observed in the live Mos cutover. The repository therefore does not, as provided, satisfy subscription-OAuth plus built-in-channel continuity.
+
+Source references:
+
+- [Repository](https://github.com/teremterem/claude-code-gpt-5-codex)
+- [Environment template](https://github.com/teremterem/claude-code-gpt-5-codex/blob/main/.env.template)
+
+## Security consequences
+
+- Subscription OAuth grants are high-value credentials and require the same lifecycle controls as service credentials.
+- Downstream virtual keys reduce provider-key exposure but do not establish user, tenant, or agent authority.
+- Automatic retry/failover can duplicate tool or external side effects unless Mosaic owns operation IDs and receipts.
+- Gateway telemetry can contain prompts, tool schemas, and model output; redaction and retention policy must apply before persistence.
+- A localhost unauthenticated translation endpoint must remain loopback-only and process-isolated.
+
+## Acceptance before production use
+
+1. Threat model and provider-terms review approved.
+2. Credential lifecycle and revocation drill documented and exercised.
+3. Adapter contract tests pass for streaming, tools, cancellation, reasoning policy, errors, and audit correlation.
+4. Tenant-bound authorization remains entirely in Mosaic Gateway.
+5. Failure injection proves no duplicate side effects across retries or provider failover.
+6. Rollback to the prior provider path is exercised.
+7. Independent code and security reviews approve the exact deployed revision.
+
+## Follow-up
+
+- #754 owns cross-harness logical identity, checkpoint, receipt, adapter, and failover work.
+- #755 / PR #757 implements the first logical identity and connector lease/fencing boundary.
+- A later issue should prototype LiteLLM and Bifrost behind the provider adapter after #755 is merged and independently qualified.

docs/tess/qualification/2026-07-14-option2-runtime-portability.md

@@ -0,0 +1,238 @@
+# Tess / Option 2 runtime-portability qualification — 2026-07-14
+
+**Issue context:** #706–#711 and runtime-neutral Mos follow-up #754
+
+**Qualified revision:** `d0771835542d` (`origin/main` at review time)
+
+**Reviewer/runtime:** Independent Pi lane requested as `openai-codex/gpt-5.6-sol:high`
+
+**Runtime resolution note:** Mosaic warned that `gpt-5.6-sol` was not present in the provider model catalog and proceeded with it as a custom model ID. This warning was part of the original qualification log and is material provenance; downstream claims must not treat catalog recognition as verified.
+
+**Verdict:** REQUEST CHANGES
+
+**Evidence type:** Point-in-time qualification; later commits and PR #757 must be reviewed separately
+
+## Purpose and provenance
+
+This report preserves the complete independent qualification that was previously available only in `/tmp/tess-option2-qualification.log`. It distinguishes passing component tests from the missing operational proof required for identity-continuous Mos failover.
+
+No credential values, OAuth tokens, Discord tokens, device codes, or auth-file contents are included. Commands and results are retained so another environment can reproduce or challenge the findings.
+
+---
+
+# 1. Verdict
+
+## **REQUEST CHANGES**
+
+The current Option 2 implementation is a useful portability foundation, but it is **not qualified against AC-TESS-01..11** and is not equivalent to true same-Mos-identity failover.
+
+Primary blockers:
+
+1. **AC-TESS-01/02:** The required `mosaic tess` command does not exist; only `mosaic interaction` is registered (`packages/mosaic/src/commands/interaction.ts:60`). The cross-surface test proves CLI enrollment followed by Discord approval/stop, not bidirectional Discord/CLI chat streaming.
+2. **AC-TESS-04:** Fleet/tmux and Matrix providers are implemented as libraries but are not registered in the production gateway. `AgentModule` registers only Hermes (`apps/gateway/src/agent/agent.module.ts:34`).
+3. **Mos handoff is not operational or durable:** Production uses `InMemoryInteractionCoordinationPort` (`apps/gateway/src/coord/coord.module.ts:18`), with no Mos-side consumer. Restart loses handoff ownership, idempotency, activity, and results.
+4. **AC-TESS-06/10:** Restart tests are good local persistence tests, but no real connector/harness failover or exercised rollback exists. Rollback is documentation-only.
+5. **AC-TESS-08:** The parity suite validates a selected shared intersection using mocked transports. Matrix is not production-wired and tmux drops the runtime message idempotency key before delivery.
+6. **AC-TESS-09:** M5 qualification remains `not-started`; no live Discord, Matrix homeserver, tmux/Mos consumer, Claude Code/Pi/Codex failover, or deployment rollback was tested.
+7. **PR #750 mismatch:** Its description promises send-error coverage as HTTP 400, but both gateway and TUI test use HTTP 403 (`packages/mosaic/src/tui/gateway-api.interaction-errors.test.ts:25-34`).
+
+### AC disposition
+
+| AC  | Result            | Evidence                                                                        |
+| --- | ----------------- | ------------------------------------------------------------------------------- |
+| 01  | **Fail**          | No `mosaic tess`; no bidirectional same-session chat/stream E2E                 |
+| 02  | **Fail**          | Generic CLI exists, but fleet/Matrix providers are unreachable in production    |
+| 03  | Pass              | Pi profile/model/reasoning/effective-policy tests passed                        |
+| 04  | **Fail**          | No registered fleet provider or real Mos consumer                               |
+| 05  | Partial           | Hermes normalization/fail-closed matrix passes; live capability path is limited |
+| 06  | Partial           | PGlite restart/idempotency passes; no actual harness failover                   |
+| 07  | Partial           | Focused denial/replay tests pass; full M5 abuse qualification absent            |
+| 08  | Partial           | Mocked shared-intersection parity passes; Matrix not operationally wired        |
+| 09  | **Fail**          | Baselines/CI green, but required E2E/security/rollback qualification absent     |
+| 10  | **Fail**          | Inventory incomplete/inconsistent; rollback not exercised                       |
+| 11  | Pass/ledger stale | Documentation and sitemap exist; plugin/catalog ledger remains unresolved       |
+
+---
+
+# 2. Exact test commands and results
+
+Initial focused attempts failed before collection because this detached worktree had no dependencies:
+
+```bash
+pnpm --filter @mosaicstack/agent exec vitest run ...
+```
+
+Result: startup failure, `Cannot find module 'vitest/config'`.
+
+Setup used:
+
+```bash
+corepack pnpm --store-dir /home/jarvis/.local/share/pnpm/store/v10 \
+  install --frozen-lockfile --ignore-scripts
+```
+
+Result: PASS, 1,240 packages linked.
+
+```bash
+corepack pnpm turbo run build \
+  --filter='@mosaicstack/gateway^...' \
+  --filter='@mosaicstack/mosaic^...'
+```
+
+Result: **17/17 dependency builds successful**.
+
+### Focused suites
+
+```bash
+corepack pnpm --filter @mosaicstack/agent exec vitest run \
+  src/runtime-provider-parity.test.ts \
+  src/matrix-native-runtime-provider.test.ts \
+  src/tmux-fleet-runtime-provider.test.ts \
+  src/durable-session.test.ts \
+  src/hermes-runtime-provider.test.ts
+```
+
+Result: **5 files, 39/39 tests passed**.
+
+```bash
+corepack pnpm --filter @mosaicstack/gateway exec vitest run \
+  src/agent/durable-session.repository.test.ts \
+  src/__tests__/integration/tess-cross-surface.integration.test.ts \
+  src/plugin/discord-ingress.security.spec.ts \
+  src/coord/interaction-coordination.service.test.ts \
+  src/coord/interaction-coordination.routing.e2e.test.ts \
+  src/agent/hermes-runtime-reachability.e2e.test.ts
+```
+
+Result: **6 files, 36/36 tests passed**. PGlite close/reopen recovery passed in 504 ms.
+
+```bash
+corepack pnpm --filter @mosaicstack/mosaic exec vitest run \
+  src/fleet/matrix-native-runtime-transport.test.ts \
+  src/fleet/tess-service-profile.test.ts \
+  src/commands/interaction.test.ts \
+  src/tui/gateway-api.interaction-errors.test.ts
+```
+
+Result: **4 files, 15/15 tests passed**.
+
+```bash
+corepack pnpm --filter @mosaicstack/coord exec vitest run \
+  src/__tests__/interaction-coordination.test.ts
+```
+
+Result: **1 file, 7/7 tests passed**.
+
+```bash
+corepack pnpm --filter @mosaicstack/gateway exec vitest run \
+  src/agent/interaction.controller.test.ts \
+  src/commands/command-authorization.service.spec.ts \
+  src/agent/__tests__/runtime-provider-registry.service.test.ts
+```
+
+Result: **3 files, 27/27 tests passed**.
+
+Focused total: **124/124 tests passed** after dependency setup.
+
+### Baselines
+
+```bash
+TURBO_FORCE=true corepack pnpm typecheck
+```
+
+Result: **42/42 tasks successful**.
+
+```bash
+TURBO_FORCE=true corepack pnpm lint
+```
+
+Result: **23/23 tasks successful**.
+
+```bash
+corepack pnpm format:check
+```
+
+Result: **PASS — all files matched Prettier style**.
+
+```bash
+~/.config/mosaic/tools/woodpecker/pipeline-status.sh \
+  -r mosaicstack/stack -n 1796
+```
+
+Result: **SUCCESS** at `d0771835542d`; all test, build, sanitization, typecheck, lint, format, and publish steps green.
+
+No tracked files outside the pre-existing `.mosaic/orchestrator/*` launcher changes were modified.
+
+---
+
+# 3. Stale ledger inconsistencies
+
+1. `docs/tess/MISSION-MANIFEST.md` still says:
+   - current milestone M1;
+   - progress 0/5;
+   - M2/M3/M5 not started.
+2. `docs/tess/TASKS.md` says:
+   - M4-V failed;
+   - M4-W-001 and TESS-PLG-001 in progress;
+   - M5-V not started.
+3. Provider issue state conflicts:
+   - #707–#709 remain open although M1–M3 rows are recorded done/pass.
+   - #710 and #711 are closed although M4-V failed and M5-V is not started.
+4. M5 work was marked done despite depending on failed M4-V.
+5. TESS-M3-002 says `mosaic tess` is done, but only `mosaic interaction` exists.
+6. PR #750 removed stale service references from operational docs, but `docs/tess/TASKS.md` still contains `MosCoordinationService` in historical notes.
+7. `docs/tess/MIGRATION-INVENTORY.md` remains an “initial inventory” with several capabilities marked `adapt`; `M5-MIGRATION-INVENTORY.md` marks grouped capabilities deferred/fail-closed. Neither supplies the complete owner/evidence matrix AC-TESS-10 requires.
+8. TESS-M2-FUP-001 remains real: the unkeyed SHA-256 compatibility branch still exists at `durable-session.repository.ts:427-431`.
+9. TESS-PLG-001 claims catalog registration was folded into W-001, but production evidence shows provider registration in the gateway—not a completed `packages/mosaic` plugin catalog.
+
+---
+
+# 4. Gap to true same-Mos-identity failover
+
+Current code can relaunch the same roster name under another runtime and can rebind a durable interaction session to another provider/runtime ID. That is **replacement**, not identity-continuous failover.
+
+Missing pieces:
+
+- No canonical logical Mos identity independent of harness-native session IDs.
+- No exclusive connector lease or monotonic fencing epoch; session rebinding is effectively last-write-wins.
+- No stale-holder rejection preventing the old harness from continuing side effects.
+- No normalized Claude Code/Pi/Codex checkpoint/import/export adapters.
+- No durable Mos coordination transport or Mos consumer.
+- No canonical handoff containing mission/task refs, git state, causal sequence, pending operations, capability requirements, and acknowledgements.
+- No end-to-end receipt journal across connectors.
+- Matrix has deterministic transaction IDs, but tmux delivery discards `RuntimeMessage.idempotencyKey`.
+- No fault-injection test transferring Mos among Claude Code, Pi, and Codex and then rolling back.
+
+---
+
+# 5. Minimal follow-up issue decomposition
+
+| Order | Issue                                       | Minimum acceptance criteria                                                                                                                                                                                                                                          |
+| ----- | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| 1     | **Logical identity and security fencing**   | Server-derived `{tenant, logicalAgentId, connectorId, harness, leaseEpoch, scopes, expiry}`; signed/fenced execution grant; stale/forged/cross-tenant grants denied and audited; no connector credential in handoffs                                                 |
+| 2     | **Durable connector lease**                 | PostgreSQL-backed exclusive lease with CAS, monotonic epoch, TTL/heartbeat, explicit takeover, and gateway rejection of stale holders; connectors for Claude Code, Pi, and Codex                                                                                     |
+| 3     | **Canonical handoff/checkpoint**            | Versioned, sealed schema containing canonical mission/task/git references, checkpoint digest, causal sequence, required capabilities, pending/ambiguous operation references, and source/destination acknowledgement; no raw secrets or mandatory harness transcript |
+| 4     | **Exactly-once connector journal**          | Durable operation IDs and receipts; idempotency propagated through every adapter; Matrix transaction mapping; tmux replaced or wrapped with receiver-side durable dedupe; ambiguous effects remain held for authorized reconciliation                                |
+| 5     | **Cross-harness failover and rollback E2E** | Real Mos identity moves Claude Code → Pi → Codex and back; inject crashes before/after lease transfer, handoff persistence, send, and acknowledgement; stale connector fenced; no duplicate side effects; canonical state preserved; rollback evidence published     |
+| 6     | **Generic gateway research ADR**            | Evaluate LiteLLM subscription OAuth and Bifrost concepts without adding either to core; include terms/security review, credential lifecycle, tenant mapping, budgets, failover semantics, and adapter-only prototype                                                 |
+
+## Generic gateway placement
+
+Allowed topology:
+
+```text
+Discord / CLI / web
+        ↓
+Mosaic Gateway: auth, tenant scope, policy, approvals, audit
+        ↓
+IProviderAdapter / AgentRuntimeProvider
+        ↓
+optional LiteLLM or Bifrost egress proxy
+        ↓
+upstream provider
+```
+
+- **LiteLLM ChatGPT subscription OAuth:** research-only, opt-in, behind an adapter. Subscription credentials require explicit terms, revocation, scope, token-storage, and audit review. They must never become Mosaic identity or core configuration.
+- **Bifrost:** virtual keys are downstream proxy credentials, not Mosaic principals. Budget and failover concepts may inform Mosaic routing, but tenant policy, authorization, and audit remain in Mosaic.
+- Neither product may introduce schemas into Mosaic core, receive direct calls from channels/agents, or bypass `IProviderAdapter`/`AgentRuntimeProvider`.
+- Mosaic should also correct its existing “all providers unhealthy → use one anyway” fallback behavior before adopting more automatic failover (`routing-engine.service.ts:204-212`).