stack/docs/tess/qualification/2026-07-14-option2-runtime-portability.md

Tess / Option 2 runtime-portability qualification — 2026-07-14

Issue context: #706–#711 and runtime-neutral Mos follow-up #754

Qualified revision: d0771835542d (origin/main at review time)

Reviewer/runtime: Independent Pi lane requested as openai-codex/gpt-5.6-sol:high

Runtime resolution note: Mosaic warned that gpt-5.6-sol was not present in the provider model catalog and proceeded with it as a custom model ID. This warning was part of the original qualification log and is material provenance; downstream claims must not treat catalog recognition as verified.

Verdict: REQUEST CHANGES

Evidence type: Point-in-time qualification; later commits and PR #757 must be reviewed separately

Purpose and provenance

This report preserves the complete independent qualification that was previously available only in /tmp/tess-option2-qualification.log. It distinguishes passing component tests from the missing operational proof required for identity-continuous Mos failover.

No credential values, OAuth tokens, Discord tokens, device codes, or auth-file contents are included. Commands and results are retained so another environment can reproduce or challenge the findings.

---

1. Verdict

REQUEST CHANGES

The current Option 2 implementation is a useful portability foundation, but it is not qualified against AC-TESS-01..11 and is not equivalent to true same-Mos-identity failover.

Primary blockers:

1. AC-TESS-01/02: The required mosaic tess command does not exist; only mosaic interaction is registered (packages/mosaic/src/commands/interaction.ts:60). The cross-surface test proves CLI enrollment followed by Discord approval/stop, not bidirectional Discord/CLI chat streaming.
2. AC-TESS-04: Fleet/tmux and Matrix providers are implemented as libraries but are not registered in the production gateway. AgentModule registers only Hermes (apps/gateway/src/agent/agent.module.ts:34).
3. Mos handoff is not operational or durable: Production uses InMemoryInteractionCoordinationPort (apps/gateway/src/coord/coord.module.ts:18), with no Mos-side consumer. Restart loses handoff ownership, idempotency, activity, and results.
4. AC-TESS-06/10: Restart tests are good local persistence tests, but no real connector/harness failover or exercised rollback exists. Rollback is documentation-only.
5. AC-TESS-08: The parity suite validates a selected shared intersection using mocked transports. Matrix is not production-wired and tmux drops the runtime message idempotency key before delivery.
6. AC-TESS-09: M5 qualification remains not-started; no live Discord, Matrix homeserver, tmux/Mos consumer, Claude Code/Pi/Codex failover, or deployment rollback was tested.
7. PR #750 mismatch: Its description promises send-error coverage as HTTP 400, but both gateway and TUI test use HTTP 403 (packages/mosaic/src/tui/gateway-api.interaction-errors.test.ts:25-34).

AC disposition

AC	Result	Evidence
01	Fail	No mosaic tess; no bidirectional same-session chat/stream E2E
02	Fail	Generic CLI exists, but fleet/Matrix providers are unreachable in production
03	Pass	Pi profile/model/reasoning/effective-policy tests passed
04	Fail	No registered fleet provider or real Mos consumer
05	Partial	Hermes normalization/fail-closed matrix passes; live capability path is limited
06	Partial	PGlite restart/idempotency passes; no actual harness failover
07	Partial	Focused denial/replay tests pass; full M5 abuse qualification absent
08	Partial	Mocked shared-intersection parity passes; Matrix not operationally wired
09	Fail	Baselines/CI green, but required E2E/security/rollback qualification absent
10	Fail	Inventory incomplete/inconsistent; rollback not exercised
11	Pass/ledger stale	Documentation and sitemap exist; plugin/catalog ledger remains unresolved

---

2. Exact test commands and results

Initial focused attempts failed before collection because this detached worktree had no dependencies:

pnpm --filter @mosaicstack/agent exec vitest run ...

Result: startup failure, Cannot find module 'vitest/config'.

Setup used:

corepack pnpm --store-dir /home/jarvis/.local/share/pnpm/store/v10 \
  install --frozen-lockfile --ignore-scripts

Result: PASS, 1,240 packages linked.

corepack pnpm turbo run build \
  --filter='@mosaicstack/gateway^...' \
  --filter='@mosaicstack/mosaic^...'

Result: 17/17 dependency builds successful.

Focused suites

corepack pnpm --filter @mosaicstack/agent exec vitest run \
  src/runtime-provider-parity.test.ts \
  src/matrix-native-runtime-provider.test.ts \
  src/tmux-fleet-runtime-provider.test.ts \
  src/durable-session.test.ts \
  src/hermes-runtime-provider.test.ts

Result: 5 files, 39/39 tests passed.

corepack pnpm --filter @mosaicstack/gateway exec vitest run \
  src/agent/durable-session.repository.test.ts \
  src/__tests__/integration/tess-cross-surface.integration.test.ts \
  src/plugin/discord-ingress.security.spec.ts \
  src/coord/interaction-coordination.service.test.ts \
  src/coord/interaction-coordination.routing.e2e.test.ts \
  src/agent/hermes-runtime-reachability.e2e.test.ts

Result: 6 files, 36/36 tests passed. PGlite close/reopen recovery passed in 504 ms.

corepack pnpm --filter @mosaicstack/mosaic exec vitest run \
  src/fleet/matrix-native-runtime-transport.test.ts \
  src/fleet/tess-service-profile.test.ts \
  src/commands/interaction.test.ts \
  src/tui/gateway-api.interaction-errors.test.ts

Result: 4 files, 15/15 tests passed.

corepack pnpm --filter @mosaicstack/coord exec vitest run \
  src/__tests__/interaction-coordination.test.ts

Result: 1 file, 7/7 tests passed.

corepack pnpm --filter @mosaicstack/gateway exec vitest run \
  src/agent/interaction.controller.test.ts \
  src/commands/command-authorization.service.spec.ts \
  src/agent/__tests__/runtime-provider-registry.service.test.ts

Result: 3 files, 27/27 tests passed.

Focused total: 124/124 tests passed after dependency setup.

Baselines

TURBO_FORCE=true corepack pnpm typecheck

Result: 42/42 tasks successful.

TURBO_FORCE=true corepack pnpm lint

Result: 23/23 tasks successful.

corepack pnpm format:check

Result: PASS — all files matched Prettier style.

~/.config/mosaic/tools/woodpecker/pipeline-status.sh \
  -r mosaicstack/stack -n 1796

Result: SUCCESS at d0771835542d; all test, build, sanitization, typecheck, lint, format, and publish steps green.

No tracked files outside the pre-existing .mosaic/orchestrator/* launcher changes were modified.

---

3. Stale ledger inconsistencies

1. docs/tess/MISSION-MANIFEST.md still says:
   • current milestone M1;
   • progress 0/5;
   • M2/M3/M5 not started.
2. docs/tess/TASKS.md says:
   • M4-V failed;
   • M4-W-001 and TESS-PLG-001 in progress;
   • M5-V not started.
3. Provider issue state conflicts:
   • #707–#709 remain open although M1–M3 rows are recorded done/pass.
   • #710 and #711 are closed although M4-V failed and M5-V is not started.
4. M5 work was marked done despite depending on failed M4-V.
5. TESS-M3-002 says mosaic tess is done, but only mosaic interaction exists.
6. PR #750 removed stale service references from operational docs, but docs/tess/TASKS.md still contains MosCoordinationService in historical notes.
7. docs/tess/MIGRATION-INVENTORY.md remains an “initial inventory” with several capabilities marked adapt; M5-MIGRATION-INVENTORY.md marks grouped capabilities deferred/fail-closed. Neither supplies the complete owner/evidence matrix AC-TESS-10 requires.
8. TESS-M2-FUP-001 remains real: the unkeyed SHA-256 compatibility branch still exists at durable-session.repository.ts:427-431.
9. TESS-PLG-001 claims catalog registration was folded into W-001, but production evidence shows provider registration in the gateway—not a completed packages/mosaic plugin catalog.

---

4. Gap to true same-Mos-identity failover

Current code can relaunch the same roster name under another runtime and can rebind a durable interaction session to another provider/runtime ID. That is replacement, not identity-continuous failover.

Missing pieces:

• No canonical logical Mos identity independent of harness-native session IDs.
• No exclusive connector lease or monotonic fencing epoch; session rebinding is effectively last-write-wins.
• No stale-holder rejection preventing the old harness from continuing side effects.
• No normalized Claude Code/Pi/Codex checkpoint/import/export adapters.
• No durable Mos coordination transport or Mos consumer.
• No canonical handoff containing mission/task refs, git state, causal sequence, pending operations, capability requirements, and acknowledgements.
• No end-to-end receipt journal across connectors.
• Matrix has deterministic transaction IDs, but tmux delivery discards RuntimeMessage.idempotencyKey.
• No fault-injection test transferring Mos among Claude Code, Pi, and Codex and then rolling back.

---

5. Minimal follow-up issue decomposition

Order	Issue	Minimum acceptance criteria
1	Logical identity and security fencing	Server-derived {tenant, logicalAgentId, connectorId, harness, leaseEpoch, scopes, expiry}; signed/fenced execution grant; stale/forged/cross-tenant grants denied and audited; no connector credential in handoffs
2	Durable connector lease	PostgreSQL-backed exclusive lease with CAS, monotonic epoch, TTL/heartbeat, explicit takeover, and gateway rejection of stale holders; connectors for Claude Code, Pi, and Codex
3	Canonical handoff/checkpoint	Versioned, sealed schema containing canonical mission/task/git references, checkpoint digest, causal sequence, required capabilities, pending/ambiguous operation references, and source/destination acknowledgement; no raw secrets or mandatory harness transcript
4	Exactly-once connector journal	Durable operation IDs and receipts; idempotency propagated through every adapter; Matrix transaction mapping; tmux replaced or wrapped with receiver-side durable dedupe; ambiguous effects remain held for authorized reconciliation
5	Cross-harness failover and rollback E2E	Real Mos identity moves Claude Code → Pi → Codex and back; inject crashes before/after lease transfer, handoff persistence, send, and acknowledgement; stale connector fenced; no duplicate side effects; canonical state preserved; rollback evidence published
6	Generic gateway research ADR	Evaluate LiteLLM subscription OAuth and Bifrost concepts without adding either to core; include terms/security review, credential lifecycle, tenant mapping, budgets, failover semantics, and adapter-only prototype

Generic gateway placement

Allowed topology:

Discord / CLI / web
        ↓
Mosaic Gateway: auth, tenant scope, policy, approvals, audit
        ↓
IProviderAdapter / AgentRuntimeProvider
        ↓
optional LiteLLM or Bifrost egress proxy
        ↓
upstream provider

• LiteLLM ChatGPT subscription OAuth: research-only, opt-in, behind an adapter. Subscription credentials require explicit terms, revocation, scope, token-storage, and audit review. They must never become Mosaic identity or core configuration.
• Bifrost: virtual keys are downstream proxy credentials, not Mosaic principals. Budget and failover concepts may inform Mosaic routing, but tenant policy, authorization, and audit remain in Mosaic.
• Neither product may introduce schemas into Mosaic core, receive direct calls from channels/agents, or bypass IProviderAdapter/AgentRuntimeProvider.
• Mosaic should also correct its existing “all providers unhealthy → use one anyway” fallback behavior before adopting more automatic failover (routing-engine.service.ts:204-212).