# Tess / Option 2 runtime-portability qualification — 2026-07-14 **Issue context:** #706–#711 and runtime-neutral Mos follow-up #754 **Qualified revision:** `d0771835542d` (`origin/main` at review time) **Reviewer/runtime:** Independent Pi lane requested as `openai-codex/gpt-5.6-sol:high` **Runtime resolution note:** Mosaic warned that `gpt-5.6-sol` was not present in the provider model catalog and proceeded with it as a custom model ID. This warning was part of the original qualification log and is material provenance; downstream claims must not treat catalog recognition as verified. **Verdict:** REQUEST CHANGES **Evidence type:** Point-in-time qualification; later commits and PR #757 must be reviewed separately ## Purpose and provenance This report preserves the complete independent qualification that was previously available only in `/tmp/tess-option2-qualification.log`. It distinguishes passing component tests from the missing operational proof required for identity-continuous Mos failover. No credential values, OAuth tokens, Discord tokens, device codes, or auth-file contents are included. Commands and results are retained so another environment can reproduce or challenge the findings. --- # 1. Verdict ## **REQUEST CHANGES** The current Option 2 implementation is a useful portability foundation, but it is **not qualified against AC-TESS-01..11** and is not equivalent to true same-Mos-identity failover. Primary blockers: 1. **AC-TESS-01/02:** The required `mosaic tess` command does not exist; only `mosaic interaction` is registered (`packages/mosaic/src/commands/interaction.ts:60`). The cross-surface test proves CLI enrollment followed by Discord approval/stop, not bidirectional Discord/CLI chat streaming. 2. **AC-TESS-04:** Fleet/tmux and Matrix providers are implemented as libraries but are not registered in the production gateway. `AgentModule` registers only Hermes (`apps/gateway/src/agent/agent.module.ts:34`). 3. **Mos handoff is not operational or durable:** Production uses `InMemoryInteractionCoordinationPort` (`apps/gateway/src/coord/coord.module.ts:18`), with no Mos-side consumer. Restart loses handoff ownership, idempotency, activity, and results. 4. **AC-TESS-06/10:** Restart tests are good local persistence tests, but no real connector/harness failover or exercised rollback exists. Rollback is documentation-only. 5. **AC-TESS-08:** The parity suite validates a selected shared intersection using mocked transports. Matrix is not production-wired and tmux drops the runtime message idempotency key before delivery. 6. **AC-TESS-09:** M5 qualification remains `not-started`; no live Discord, Matrix homeserver, tmux/Mos consumer, Claude Code/Pi/Codex failover, or deployment rollback was tested. 7. **PR #750 mismatch:** Its description promises send-error coverage as HTTP 400, but both gateway and TUI test use HTTP 403 (`packages/mosaic/src/tui/gateway-api.interaction-errors.test.ts:25-34`). ### AC disposition | AC | Result | Evidence | | --- | ----------------- | ------------------------------------------------------------------------------- | | 01 | **Fail** | No `mosaic tess`; no bidirectional same-session chat/stream E2E | | 02 | **Fail** | Generic CLI exists, but fleet/Matrix providers are unreachable in production | | 03 | Pass | Pi profile/model/reasoning/effective-policy tests passed | | 04 | **Fail** | No registered fleet provider or real Mos consumer | | 05 | Partial | Hermes normalization/fail-closed matrix passes; live capability path is limited | | 06 | Partial | PGlite restart/idempotency passes; no actual harness failover | | 07 | Partial | Focused denial/replay tests pass; full M5 abuse qualification absent | | 08 | Partial | Mocked shared-intersection parity passes; Matrix not operationally wired | | 09 | **Fail** | Baselines/CI green, but required E2E/security/rollback qualification absent | | 10 | **Fail** | Inventory incomplete/inconsistent; rollback not exercised | | 11 | Pass/ledger stale | Documentation and sitemap exist; plugin/catalog ledger remains unresolved | --- # 2. Exact test commands and results Initial focused attempts failed before collection because this detached worktree had no dependencies: ```bash pnpm --filter @mosaicstack/agent exec vitest run ... ``` Result: startup failure, `Cannot find module 'vitest/config'`. Setup used: ```bash corepack pnpm --store-dir /home/jarvis/.local/share/pnpm/store/v10 \ install --frozen-lockfile --ignore-scripts ``` Result: PASS, 1,240 packages linked. ```bash corepack pnpm turbo run build \ --filter='@mosaicstack/gateway^...' \ --filter='@mosaicstack/mosaic^...' ``` Result: **17/17 dependency builds successful**. ### Focused suites ```bash corepack pnpm --filter @mosaicstack/agent exec vitest run \ src/runtime-provider-parity.test.ts \ src/matrix-native-runtime-provider.test.ts \ src/tmux-fleet-runtime-provider.test.ts \ src/durable-session.test.ts \ src/hermes-runtime-provider.test.ts ``` Result: **5 files, 39/39 tests passed**. ```bash corepack pnpm --filter @mosaicstack/gateway exec vitest run \ src/agent/durable-session.repository.test.ts \ src/__tests__/integration/tess-cross-surface.integration.test.ts \ src/plugin/discord-ingress.security.spec.ts \ src/coord/interaction-coordination.service.test.ts \ src/coord/interaction-coordination.routing.e2e.test.ts \ src/agent/hermes-runtime-reachability.e2e.test.ts ``` Result: **6 files, 36/36 tests passed**. PGlite close/reopen recovery passed in 504 ms. ```bash corepack pnpm --filter @mosaicstack/mosaic exec vitest run \ src/fleet/matrix-native-runtime-transport.test.ts \ src/fleet/tess-service-profile.test.ts \ src/commands/interaction.test.ts \ src/tui/gateway-api.interaction-errors.test.ts ``` Result: **4 files, 15/15 tests passed**. ```bash corepack pnpm --filter @mosaicstack/coord exec vitest run \ src/__tests__/interaction-coordination.test.ts ``` Result: **1 file, 7/7 tests passed**. ```bash corepack pnpm --filter @mosaicstack/gateway exec vitest run \ src/agent/interaction.controller.test.ts \ src/commands/command-authorization.service.spec.ts \ src/agent/__tests__/runtime-provider-registry.service.test.ts ``` Result: **3 files, 27/27 tests passed**. Focused total: **124/124 tests passed** after dependency setup. ### Baselines ```bash TURBO_FORCE=true corepack pnpm typecheck ``` Result: **42/42 tasks successful**. ```bash TURBO_FORCE=true corepack pnpm lint ``` Result: **23/23 tasks successful**. ```bash corepack pnpm format:check ``` Result: **PASS — all files matched Prettier style**. ```bash ~/.config/mosaic/tools/woodpecker/pipeline-status.sh \ -r mosaicstack/stack -n 1796 ``` Result: **SUCCESS** at `d0771835542d`; all test, build, sanitization, typecheck, lint, format, and publish steps green. No tracked files outside the pre-existing `.mosaic/orchestrator/*` launcher changes were modified. --- # 3. Stale ledger inconsistencies 1. `docs/tess/MISSION-MANIFEST.md` still says: - current milestone M1; - progress 0/5; - M2/M3/M5 not started. 2. `docs/tess/TASKS.md` says: - M4-V failed; - M4-W-001 and TESS-PLG-001 in progress; - M5-V not started. 3. Provider issue state conflicts: - #707–#709 remain open although M1–M3 rows are recorded done/pass. - #710 and #711 are closed although M4-V failed and M5-V is not started. 4. M5 work was marked done despite depending on failed M4-V. 5. TESS-M3-002 says `mosaic tess` is done, but only `mosaic interaction` exists. 6. PR #750 removed stale service references from operational docs, but `docs/tess/TASKS.md` still contains `MosCoordinationService` in historical notes. 7. `docs/tess/MIGRATION-INVENTORY.md` remains an “initial inventory” with several capabilities marked `adapt`; `M5-MIGRATION-INVENTORY.md` marks grouped capabilities deferred/fail-closed. Neither supplies the complete owner/evidence matrix AC-TESS-10 requires. 8. TESS-M2-FUP-001 remains real: the unkeyed SHA-256 compatibility branch still exists at `durable-session.repository.ts:427-431`. 9. TESS-PLG-001 claims catalog registration was folded into W-001, but production evidence shows provider registration in the gateway—not a completed `packages/mosaic` plugin catalog. --- # 4. Gap to true same-Mos-identity failover Current code can relaunch the same roster name under another runtime and can rebind a durable interaction session to another provider/runtime ID. That is **replacement**, not identity-continuous failover. Missing pieces: - No canonical logical Mos identity independent of harness-native session IDs. - No exclusive connector lease or monotonic fencing epoch; session rebinding is effectively last-write-wins. - No stale-holder rejection preventing the old harness from continuing side effects. - No normalized Claude Code/Pi/Codex checkpoint/import/export adapters. - No durable Mos coordination transport or Mos consumer. - No canonical handoff containing mission/task refs, git state, causal sequence, pending operations, capability requirements, and acknowledgements. - No end-to-end receipt journal across connectors. - Matrix has deterministic transaction IDs, but tmux delivery discards `RuntimeMessage.idempotencyKey`. - No fault-injection test transferring Mos among Claude Code, Pi, and Codex and then rolling back. --- # 5. Minimal follow-up issue decomposition | Order | Issue | Minimum acceptance criteria | | ----- | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | **Logical identity and security fencing** | Server-derived `{tenant, logicalAgentId, connectorId, harness, leaseEpoch, scopes, expiry}`; signed/fenced execution grant; stale/forged/cross-tenant grants denied and audited; no connector credential in handoffs | | 2 | **Durable connector lease** | PostgreSQL-backed exclusive lease with CAS, monotonic epoch, TTL/heartbeat, explicit takeover, and gateway rejection of stale holders; connectors for Claude Code, Pi, and Codex | | 3 | **Canonical handoff/checkpoint** | Versioned, sealed schema containing canonical mission/task/git references, checkpoint digest, causal sequence, required capabilities, pending/ambiguous operation references, and source/destination acknowledgement; no raw secrets or mandatory harness transcript | | 4 | **Exactly-once connector journal** | Durable operation IDs and receipts; idempotency propagated through every adapter; Matrix transaction mapping; tmux replaced or wrapped with receiver-side durable dedupe; ambiguous effects remain held for authorized reconciliation | | 5 | **Cross-harness failover and rollback E2E** | Real Mos identity moves Claude Code → Pi → Codex and back; inject crashes before/after lease transfer, handoff persistence, send, and acknowledgement; stale connector fenced; no duplicate side effects; canonical state preserved; rollback evidence published | | 6 | **Generic gateway research ADR** | Evaluate LiteLLM subscription OAuth and Bifrost concepts without adding either to core; include terms/security review, credential lifecycle, tenant mapping, budgets, failover semantics, and adapter-only prototype | ## Generic gateway placement Allowed topology: ```text Discord / CLI / web ↓ Mosaic Gateway: auth, tenant scope, policy, approvals, audit ↓ IProviderAdapter / AgentRuntimeProvider ↓ optional LiteLLM or Bifrost egress proxy ↓ upstream provider ``` - **LiteLLM ChatGPT subscription OAuth:** research-only, opt-in, behind an adapter. Subscription credentials require explicit terms, revocation, scope, token-storage, and audit review. They must never become Mosaic identity or core configuration. - **Bifrost:** virtual keys are downstream proxy credentials, not Mosaic principals. Budget and failover concepts may inform Mosaic routing, but tenant policy, authorization, and audit remain in Mosaic. - Neither product may introduce schemas into Mosaic core, receive direct calls from channels/agents, or bypass `IProviderAdapter`/`AgentRuntimeProvider`. - Mosaic should also correct its existing “all providers unhealthy → use one anyway” fallback behavior before adopting more automatic failover (`routing-engine.service.ts:204-212`).