stack/docs/DEVELOPER-GUIDE/architecture/rfcs/optional-ai-egress-gateways.md

RFC: Optional AI Egress Gateways

Status: Draft / proposed — not approved, not current, and not integrated.

Authority: Non-operative design proposal. This document does not authorize an integration, deployment, provider selection, credential flow, or production use. It is intentionally an RFC rather than an approved architecture decision.

Scope: Optional model/inference egress only. IProviderAdapter and AgentRuntimeProvider are separate boundaries; this RFC does not merge them.

• Date: 2026-07-14
• Related issues: #754, #755
• Decision owner: Mosaic Gateway / provider-adapter architecture

Context

The emergency Mos continuity path kept Claude Code as the harness and translated Anthropic Messages traffic to Codex OAuth through a small localhost proxy. That preserved the existing Claude Discord plugin and transcript, but exposed two architectural facts:

1. Harness identity, channel entitlement, provider credentials, inference transport, and runtime sessions are separate concerns.
2. A generic AI gateway could eventually improve model routing, budgets, and observability, but it must not become Mosaic's identity, authorization, tenant, connector, or orchestration boundary.

The Tess qualification work also found that provider rebinding is not, by itself, identity-continuous failover. Any future design still needs a logical agent identity, durable connector lease/fencing, canonical handoff/checkpoint, exactly-once receipts, concrete runtime adapters, and cross-runtime rollback validation.

This page is a reclassified migration of the historical egress-gateway proposal. The move to docs/DEVELOPER-GUIDE/architecture/rfcs/ does not promote it to a decision and does not change runtime behavior.

Current status and non-goals

This RFC describes a possible future integration boundary. It does not say that any of the candidates below is supported by Mosaic Gateway today:

• LiteLLM is not integrated. It is only a candidate for a future model-egress adapter and remains subject to the prerequisites in this RFC.
• Bifrost is not integrated. Its routing, virtual-key, and failover features are research inputs only, not Mosaic authority.
• Claudex and claude-code-proxy are experimental harness tooling, not gateway egress. The mosaic claudex path runs a Claude Code harness against a local translation proxy for evaluation. It is not a Mosaic Gateway egress integration, does not define the IProviderAdapter model contract, and does not define the AgentRuntimeProvider runtime/session contract.

In particular, this RFC does not propose that a generic gateway receive channel traffic directly, own Mosaic sessions, replace the runtime-provider registry, or become a second authorization or tenant system. No database, service, or runtime change is implied by this document.

Corrected contract boundary

The historical proposal conflated two different adapter families. They must remain separate.

IProviderAdapter: model and inference egress

IProviderAdapter is the model-provider abstraction. Its current contract covers provider registration, model discovery, provider health, and a future direct completion stream. ProviderService aggregates these adapters and connects model availability to the Pi ModelRegistry.

It does not represent an agent process or session. It does not own channel identity, Mosaic actor or tenant identity, logical-agent ownership, runtime session IDs, runtime attachments, handoff state, or tool authorization. An optional LiteLLM, Bifrost, or similar model gateway would therefore be a candidate for this model-egress path only, after approval and qualification.

AgentRuntimeProvider: runtime and session boundary

AgentRuntimeProvider is the runtime-neutral session contract. It covers runtime capabilities and health plus operations such as listing sessions, reading session trees, streaming a session, sending a message, attaching or detaching, and terminating a session. Its RuntimeScope is derived from trusted actor, tenant, channel, and correlation context.

The runtime-provider service performs the gateway-side capability, authorization, approval, and metadata-only audit boundary before invoking a runtime provider. It is not a model-egress gateway and must not be substituted for IProviderAdapter. A runtime implementation may use model selection or a model service internally, but that dependency is an explicit composition between two contracts; it does not make the runtime provider a model provider or make a model gateway a session provider.

The checked-in contract and implementation references are:

• IProviderAdapter and provider types
• ProviderService
• AgentRuntimeProvider
• RuntimeProviderService
• AgentRuntimeProviderRegistry

Proposed relationship

If a future implementation is approved, the model and runtime paths remain parallel and explicitly composed at the Mosaic boundary:

Discord / Matrix / CLI / web
             |
             v
Mosaic Gateway: authenticated actor + tenant, policy, approvals,
               logical agent, connector lease/fence, audit, checkpoints,
               idempotency, and side-effect receipts
             |
       +-----+-----------------------+
       |                             |
       v                             v
model request path              runtime/session path
ProviderService / routing      RuntimeProviderService
       |                             |
       v                             v
IProviderAdapter                AgentRuntimeProvider
       |                             |
       v                             v
optional model-egress           native or external runtime/session
gateway (future only)           transport
       |
       v
upstream model/provider

The optional egress gateway may receive only a validated model request from the Mosaic model path. It is not a channel ingress, runtime-session transport, connector owner, or source of Mosaic identity.

Draft proposal

This RFC proposes, for review only, that Mosaic could support an optional model-egress gateway through a dedicated IProviderAdapter implementation or adapter-owned model transport. Any such integration would remain subordinate to Mosaic Gateway policy and would not add a second runtime/session boundary.

Mosaic Gateway would remain authoritative for:

• authenticated actor and tenant identity;
• logical-agent identity, connector binding, lease epoch, and stale-holder fencing;
• authorization, approval, and model/tool policy;
• runtime/session access through the separate AgentRuntimeProvider boundary;
• audit correlation, redaction, retention, and operational evidence;
• canonical handoff, checkpoint, and recovery state; and
• idempotency keys, operation identity, and durable side-effect receipts.

An optional model-egress gateway must not:

• receive channel ingress directly;
• authorize tools, connector ownership, runtime sessions, or approvals;
• define Mosaic actors, tenants, logical agents, or session identity;
• treat downstream virtual keys as Mosaic principals;
• persist raw Mosaic handoffs, channel credentials, or unredacted telemetry;
• bypass adapter capability negotiation or gateway policy;
• retry or fail over an operation whose side-effect state is ambiguous; or
• silently select an unhealthy or unauthorized provider merely to return a result.

Candidate assessment

These dispositions are deliberately non-integrated and do not constitute approval.

LiteLLM

Disposition: Not integrated; future candidate for a formal IProviderAdapter model-egress prototype only.

Potentially useful features include broad provider routing, virtual keys, budgets, observability, and OpenAI/Anthropic-compatible surfaces. A future review must verify, rather than assume, the exact ChatGPT subscription OAuth flow, supported models, provider terms, token storage, encryption, revocation, refresh, scope, and incident response.

A prototype would also have to prove streaming, tool calls, reasoning controls, cancellation, tenant isolation, audit-correlation preservation, retry behavior, and idempotency. Virtual keys must remain downstream credentials and must not become Mosaic actors or tenants. Channel ingress and connector credentials would remain outside LiteLLM.

Source references:

• LiteLLM ChatGPT subscription provider
• LiteLLM providers

Bifrost

Disposition: Not integrated; future candidate for governance and routing research, with subscription OAuth compatibility unverified.

Virtual keys, budgets, rate limits, weighted load balancing, and provider failover may inform a future Mosaic egress design, but they are not Mosaic authority. A future review must verify Codex/ChatGPT subscription OAuth rather than assume API-key compatibility, map all policy to server-derived Mosaic tenants, prove that failover preserves connector leases and approvals, and redact request/response telemetry before persistence.

Automatic failover must be disabled or constrained whenever policy, approval, or side-effect state is ambiguous.

Source references:

• Bifrost overview
• Bifrost repository

Claudex / claude-code-proxy

Disposition: Experimental harness overlay and local translation proxy; not Mosaic Gateway egress and not an approved provider integration.

The reviewed path runs GPT models inside the Claude Code harness through a local claude-code-proxy that translates Anthropic Messages traffic to a ChatGPT-subscription (Codex OAuth) backend. It is intended for evaluation and must not be described as a current Mosaic model gateway, an IProviderAdapter implementation, or an AgentRuntimeProvider implementation.

Its constraints remain important: it is not a multi-tenant control plane, it must not own connector leasing or canonical handoff, and local proxy credentials and listener exposure require isolation. The current Mosaic launcher documents its experimental status and isolated Claude configuration:

• mosaic claudex documentation
• claudex launch composition
• claude-code-proxy

teremterem/claude-code-gpt-5-codex

Disposition: Historical recipe; not selected and not integrated.

The reviewed repository uses OPENAI_API_KEY, tells previously authenticated Claude users to log out, and documents a Claude Web Search schema incompatibility. That does not satisfy the subscription-OAuth plus built-in- channel continuity requirement observed in the Mos cutover.

Source references:

• Repository
• Environment template

Prerequisites before approval or integration

No candidate may be integrated, enabled as a Mosaic feature, or treated as a current supported path until all of the following are complete for the exact provider, adapter, configuration, and deployed revision.

1. Recorded approval and governance

• An architecture decision explicitly approves the model-egress scope and records that IProviderAdapter remains separate from AgentRuntimeProvider.
• Product/operations ownership, provider terms review, and independent security review are recorded against the exact change and provider revision.
• The approval identifies allowed providers, models, regions, data handling, rollback owner, support status, and a time-bounded experimental or production phase. This RFC itself is not that approval.
• No candidate is advertised as integrated, current, or production-ready while the approval record is absent or expired.

2. Security and credential controls

• A threat model covers prompt/tool data, model output, streaming, cancellation, retries, failover, SSRF, endpoint authentication, TLS, egress allowlists, supply-chain risk, provider terms, and denial-of-service behavior.
• OAuth tokens, API keys, virtual keys, refresh tokens, and proxy credentials have documented ownership, storage, encryption, rotation, revocation, expiry, least-privilege scope, and incident-response procedures. Secrets do not enter prompts, logs, traces, audit payloads, or commits.
• Request, response, tool-schema, and error telemetry is classified and redacted before persistence or external channel delivery. Retention and deletion are tenant-scoped.
• The adapter fails closed on missing, expired, revoked, malformed, or unauthorized credentials and on uncertain provider health. Loopback-only experimental proxies remain process-isolated and cannot become a gateway bypass.

3. Actor and tenant isolation

• Actor and tenant identity are derived from authenticated Mosaic context at the Gateway; callers cannot select a tenant, logical agent, credential, or provider principal by supplying an ID.
• Downstream virtual keys and provider account identifiers are mapped to server-owned tenant policy. They never grant Mosaic authorization and never replace RBAC, approvals, connector leases, or runtime scope.
• Model configuration, budgets, rate limits, data residency, credential use, logs, caches, and failure handling are isolated per tenant. Cross-tenant reads, writes, cache hits, telemetry, and failover are denied and tested.
• The egress adapter receives only the minimum scoped request needed for the authorized model operation; channel ingress and runtime/session control stay in Mosaic-owned boundaries.

4. Idempotency and side-effect safety

• Mosaic assigns a durable operation ID and idempotency key before an egress request can cause a tool or external side effect. The provider gateway must preserve the correlation and idempotency metadata or be wrapped by an adapter that does so.
• Durable receipts record request, attempt, provider, outcome, and replay state without storing unredacted content. Retries and failover are allowed only when the operation contract proves they cannot duplicate a side effect.
• Ambiguous timeout, disconnect, stream-resume, cancellation, and provider failover outcomes fail closed until the receipt is reconciled. The egress gateway must not claim exactly-once behavior that Mosaic has not proven.
• Failure injection demonstrates no duplicate tool, connector, channel, or external side effect across gateway, adapter, proxy, and provider retries.

5. Contract and rollback qualification

• Separate contract tests pass for IProviderAdapter model registration, health, streaming, tools, reasoning controls, cancellation, errors, and audit correlation.
• Separate AgentRuntimeProvider contract tests continue to pass for runtime capability negotiation, session ownership, approval, attach/detach, termination, and normalized events. A model-egress candidate must not be used as evidence for runtime/session compatibility.
• A verified rollback to the prior model path is exercised, including revoked credentials, unhealthy providers, partial streams, and an adapter version mismatch.
• The exact deployed revision receives independent code and security review, with terminal-green CI and operator recovery evidence before any activation.

Security consequences

• Subscription OAuth grants are high-value credentials and require the same lifecycle controls as service credentials.
• Downstream virtual keys reduce provider-key exposure but do not establish user, tenant, agent, or runtime authority.
• Automatic retry or failover can duplicate tool and external side effects unless Mosaic owns operation IDs, durable receipts, and reconciliation.
• Gateway telemetry can contain prompts, tool schemas, and model output; redaction and retention policy must apply before persistence.
• A localhost translation endpoint must remain loopback-only, process-isolated, authenticated where applicable, and outside the Mosaic Gateway ingress path.
• A model-egress gateway outage must not weaken Mosaic authorization, lease fencing, tenant isolation, approval, or runtime-session boundaries.

Acceptance before any production use

The following are minimum exit criteria for a future, separately approved implementation; they are not satisfied by this RFC:

1. The recorded approval and provider-terms review cover the exact integration.
2. Credential lifecycle, revocation, rotation, redaction, and incident drills are documented and exercised.
3. Tenant-bound authorization remains entirely in Mosaic Gateway and passes cross-tenant negative tests.
4. IProviderAdapter contract tests pass without treating AgentRuntimeProvider tests as substitutes.
5. Failure injection proves no duplicate side effects across retries or provider failover, with durable idempotency receipts.
6. Streaming, tools, cancellation, reasoning policy, errors, and audit correlation are qualified for the exact model path.
7. Rollback to the prior provider path is exercised and operator-verifiable.
8. Independent code and security reviews approve the exact deployed revision.

Follow-up

• #754 owns cross-runtime logical identity, checkpoint, receipt, adapter, and failover work.
• #755 / PR #757 implements the first logical identity and connector lease/fencing boundary.
• A later issue may prototype LiteLLM or Bifrost behind the model-provider boundary only after the recorded approval, security, tenant, idempotency, contract, and rollback prerequisites are met.
• The page remains Draft / proposed until an explicit decision changes its status. Until then, it must not be cited as an approved architecture, current integration, or operational runbook.