---
kind: spec
status: active
---

# RFC: Optional AI Egress Gateways

> **Status:** Draft / proposed — not approved, not current, and not integrated.
>
> **Authority:** Non-operative design proposal. This document does not authorize an
> integration, deployment, provider selection, credential flow, or production use.
> It is intentionally an RFC rather than an approved architecture decision.
>
> **Scope:** Optional model/inference egress only. `IProviderAdapter` and
> `AgentRuntimeProvider` are separate boundaries; this RFC does not merge them.

- **Date:** 2026-07-14
- **Related issues:** #754, #755
- **Decision owner:** Mosaic Gateway / provider-adapter architecture

## Context

The emergency Mos continuity path kept Claude Code as the harness and translated
Anthropic Messages traffic to Codex OAuth through a small localhost proxy. That
preserved the existing Claude Discord plugin and transcript, but exposed two
architectural facts:

1. Harness identity, channel entitlement, provider credentials, inference
   transport, and runtime sessions are separate concerns.
2. A generic AI gateway could eventually improve model routing, budgets, and
   observability, but it must not become Mosaic's identity, authorization,
   tenant, connector, or orchestration boundary.

The Tess qualification work also found that provider rebinding is not, by
itself, identity-continuous failover. Any future design still needs a logical
agent identity, durable connector lease/fencing, canonical handoff/checkpoint,
exactly-once receipts, concrete runtime adapters, and cross-runtime rollback
validation.

This page is a reclassified migration of the historical egress-gateway
proposal. The move to `docs/DEVELOPER-GUIDE/architecture/rfcs/` does not promote
it to a decision and does not change runtime behavior.

## Current status and non-goals

This RFC describes a possible future integration boundary. It does **not** say
that any of the candidates below is supported by Mosaic Gateway today:

- **LiteLLM is not integrated.** It is only a candidate for a future model-egress
  adapter and remains subject to the prerequisites in this RFC.
- **Bifrost is not integrated.** Its routing, virtual-key, and failover features
  are research inputs only, not Mosaic authority.
- **Claudex and `claude-code-proxy` are experimental harness tooling, not
  gateway egress.** The `mosaic claudex` path runs a Claude Code harness against
  a local translation proxy for evaluation. It is not a Mosaic Gateway egress
  integration, does not define the `IProviderAdapter` model contract, and does
  not define the `AgentRuntimeProvider` runtime/session contract.

In particular, this RFC does not propose that a generic gateway receive channel
traffic directly, own Mosaic sessions, replace the runtime-provider registry,
or become a second authorization or tenant system. No database, service, or
runtime change is implied by this document.

## Corrected contract boundary

The historical proposal conflated two different adapter families. They must
remain separate.

### `IProviderAdapter`: model and inference egress

`IProviderAdapter` is the model-provider abstraction. Its current contract
covers provider registration, model discovery, provider health, and a future
direct completion stream. `ProviderService` aggregates these adapters and
connects model availability to the Pi `ModelRegistry`.

It does **not** represent an agent process or session. It does not own channel
identity, Mosaic actor or tenant identity, logical-agent ownership, runtime
session IDs, runtime attachments, handoff state, or tool authorization. An
optional LiteLLM, Bifrost, or similar model gateway would therefore be a
candidate for this model-egress path only, after approval and qualification.

### `AgentRuntimeProvider`: runtime and session boundary

`AgentRuntimeProvider` is the runtime-neutral session contract. It covers
runtime capabilities and health plus operations such as listing sessions,
reading session trees, streaming a session, sending a message, attaching or
detaching, and terminating a session. Its `RuntimeScope` is derived from
trusted actor, tenant, channel, and correlation context.

The runtime-provider service performs the gateway-side capability, authorization,
approval, and metadata-only audit boundary before invoking a runtime provider.
It is not a model-egress gateway and must not be substituted for
`IProviderAdapter`. A runtime implementation may use model selection or a model
service internally, but that dependency is an explicit composition between two
contracts; it does not make the runtime provider a model provider or make a
model gateway a session provider.

The checked-in contract and implementation references are:

- [`IProviderAdapter` and provider types](../../../../packages/types/src/provider/index.ts)
- [`ProviderService`](../../../../apps/gateway/src/agent/provider.service.ts)
- [`AgentRuntimeProvider`](../../../../packages/types/src/agent/agent-runtime-provider.ts)
- [`RuntimeProviderService`](../../../../apps/gateway/src/agent/runtime-provider-registry.service.ts)
- [`AgentRuntimeProviderRegistry`](../../../../packages/agent/src/runtime-provider-registry.ts)

### Proposed relationship

If a future implementation is approved, the model and runtime paths remain
parallel and explicitly composed at the Mosaic boundary:

```text
Discord / Matrix / CLI / web
             |
             v
Mosaic Gateway: authenticated actor + tenant, policy, approvals,
               logical agent, connector lease/fence, audit, checkpoints,
               idempotency, and side-effect receipts
             |
       +-----+-----------------------+
       |                             |
       v                             v
model request path              runtime/session path
ProviderService / routing      RuntimeProviderService
       |                             |
       v                             v
IProviderAdapter                AgentRuntimeProvider
       |                             |
       v                             v
optional model-egress           native or external runtime/session
gateway (future only)           transport
       |
       v
upstream model/provider
```

The optional egress gateway may receive only a validated model request from the
Mosaic model path. It is not a channel ingress, runtime-session transport,
connector owner, or source of Mosaic identity.

## Draft proposal

This RFC proposes, for review only, that Mosaic could support an optional
model-egress gateway through a dedicated `IProviderAdapter` implementation or
adapter-owned model transport. Any such integration would remain subordinate
to Mosaic Gateway policy and would not add a second runtime/session boundary.

Mosaic Gateway would remain authoritative for:

- authenticated actor and tenant identity;
- logical-agent identity, connector binding, lease epoch, and stale-holder
  fencing;
- authorization, approval, and model/tool policy;
- runtime/session access through the separate `AgentRuntimeProvider` boundary;
- audit correlation, redaction, retention, and operational evidence;
- canonical handoff, checkpoint, and recovery state; and
- idempotency keys, operation identity, and durable side-effect receipts.

An optional model-egress gateway must not:

- receive channel ingress directly;
- authorize tools, connector ownership, runtime sessions, or approvals;
- define Mosaic actors, tenants, logical agents, or session identity;
- treat downstream virtual keys as Mosaic principals;
- persist raw Mosaic handoffs, channel credentials, or unredacted telemetry;
- bypass adapter capability negotiation or gateway policy;
- retry or fail over an operation whose side-effect state is ambiguous; or
- silently select an unhealthy or unauthorized provider merely to return a
  result.

## Candidate assessment

These dispositions are deliberately non-integrated and do not constitute
approval.

### LiteLLM

**Disposition:** Not integrated; future candidate for a formal
`IProviderAdapter` model-egress prototype only.

Potentially useful features include broad provider routing, virtual keys,
budgets, observability, and OpenAI/Anthropic-compatible surfaces. A future
review must verify, rather than assume, the exact ChatGPT subscription OAuth
flow, supported models, provider terms, token storage, encryption, revocation,
refresh, scope, and incident response.

A prototype would also have to prove streaming, tool calls, reasoning controls,
cancellation, tenant isolation, audit-correlation preservation, retry behavior,
and idempotency. Virtual keys must remain downstream credentials and must not
become Mosaic actors or tenants. Channel ingress and connector credentials
would remain outside LiteLLM.

Source references:

- [LiteLLM ChatGPT subscription provider](https://docs.litellm.ai/docs/providers/chatgpt)
- [LiteLLM providers](https://docs.litellm.ai/docs/providers)

### Bifrost

**Disposition:** Not integrated; future candidate for governance and routing
research, with subscription OAuth compatibility unverified.

Virtual keys, budgets, rate limits, weighted load balancing, and provider
failover may inform a future Mosaic egress design, but they are not Mosaic
authority. A future review must verify Codex/ChatGPT subscription OAuth rather
than assume API-key compatibility, map all policy to server-derived Mosaic
tenants, prove that failover preserves connector leases and approvals, and
redact request/response telemetry before persistence.

Automatic failover must be disabled or constrained whenever policy, approval,
or side-effect state is ambiguous.

Source references:

- [Bifrost overview](https://docs.getbifrost.ai/overview)
- [Bifrost repository](https://github.com/maximhq/bifrost)

### Claudex / `claude-code-proxy`

**Disposition:** Experimental harness overlay and local translation proxy; not
Mosaic Gateway egress and not an approved provider integration.

The reviewed path runs GPT models inside the Claude Code harness through a
local `claude-code-proxy` that translates Anthropic Messages traffic to a
ChatGPT-subscription (Codex OAuth) backend. It is intended for evaluation and
must not be described as a current Mosaic model gateway, an `IProviderAdapter`
implementation, or an `AgentRuntimeProvider` implementation.

Its constraints remain important: it is not a multi-tenant control plane, it
must not own connector leasing or canonical handoff, and local proxy
credentials and listener exposure require isolation. The current Mosaic
launcher documents its experimental status and isolated Claude configuration:

- [`mosaic` claudex documentation](../../../../packages/mosaic/README.md)
- [`claudex` launch composition](../../../../packages/mosaic/src/commands/claudex.ts)
- [`claude-code-proxy`](https://github.com/raine/claude-code-proxy)

### `teremterem/claude-code-gpt-5-codex`

**Disposition:** Historical recipe; not selected and not integrated.

The reviewed repository uses `OPENAI_API_KEY`, tells previously authenticated
Claude users to log out, and documents a Claude Web Search schema
incompatibility. That does not satisfy the subscription-OAuth plus built-in-
channel continuity requirement observed in the Mos cutover.

Source references:

- [Repository](https://github.com/teremterem/claude-code-gpt-5-codex)
- [Environment template](https://github.com/teremterem/claude-code-gpt-5-codex/blob/main/.env.template)

## Prerequisites before approval or integration

No candidate may be integrated, enabled as a Mosaic feature, or treated as a
current supported path until all of the following are complete for the exact
provider, adapter, configuration, and deployed revision.

### 1. Recorded approval and governance

- An architecture decision explicitly approves the model-egress scope and
  records that `IProviderAdapter` remains separate from `AgentRuntimeProvider`.
- Product/operations ownership, provider terms review, and independent security
  review are recorded against the exact change and provider revision.
- The approval identifies allowed providers, models, regions, data handling,
  rollback owner, support status, and a time-bounded experimental or production
  phase. This RFC itself is not that approval.
- No candidate is advertised as integrated, current, or production-ready while
  the approval record is absent or expired.

### 2. Security and credential controls

- A threat model covers prompt/tool data, model output, streaming, cancellation,
  retries, failover, SSRF, endpoint authentication, TLS, egress allowlists,
  supply-chain risk, provider terms, and denial-of-service behavior.
- OAuth tokens, API keys, virtual keys, refresh tokens, and proxy credentials
  have documented ownership, storage, encryption, rotation, revocation,
  expiry, least-privilege scope, and incident-response procedures. Secrets do
  not enter prompts, logs, traces, audit payloads, or commits.
- Request, response, tool-schema, and error telemetry is classified and
  redacted before persistence or external channel delivery. Retention and
  deletion are tenant-scoped.
- The adapter fails closed on missing, expired, revoked, malformed, or
  unauthorized credentials and on uncertain provider health. Loopback-only
  experimental proxies remain process-isolated and cannot become a gateway
  bypass.

### 3. Actor and tenant isolation

- Actor and tenant identity are derived from authenticated Mosaic context at
  the Gateway; callers cannot select a tenant, logical agent, credential, or
  provider principal by supplying an ID.
- Downstream virtual keys and provider account identifiers are mapped to
  server-owned tenant policy. They never grant Mosaic authorization and never
  replace RBAC, approvals, connector leases, or runtime scope.
- Model configuration, budgets, rate limits, data residency, credential use,
  logs, caches, and failure handling are isolated per tenant. Cross-tenant
  reads, writes, cache hits, telemetry, and failover are denied and tested.
- The egress adapter receives only the minimum scoped request needed for the
  authorized model operation; channel ingress and runtime/session control stay
  in Mosaic-owned boundaries.

### 4. Idempotency and side-effect safety

- Mosaic assigns a durable operation ID and idempotency key before an egress
  request can cause a tool or external side effect. The provider gateway must
  preserve the correlation and idempotency metadata or be wrapped by an
  adapter that does so.
- Durable receipts record request, attempt, provider, outcome, and replay state
  without storing unredacted content. Retries and failover are allowed only
  when the operation contract proves they cannot duplicate a side effect.
- Ambiguous timeout, disconnect, stream-resume, cancellation, and provider
  failover outcomes fail closed until the receipt is reconciled. The egress
  gateway must not claim exactly-once behavior that Mosaic has not proven.
- Failure injection demonstrates no duplicate tool, connector, channel, or
  external side effect across gateway, adapter, proxy, and provider retries.

### 5. Contract and rollback qualification

- Separate contract tests pass for `IProviderAdapter` model registration,
  health, streaming, tools, reasoning controls, cancellation, errors, and
  audit correlation.
- Separate `AgentRuntimeProvider` contract tests continue to pass for runtime
  capability negotiation, session ownership, approval, attach/detach,
  termination, and normalized events. A model-egress candidate must not be
  used as evidence for runtime/session compatibility.
- A verified rollback to the prior model path is exercised, including revoked
  credentials, unhealthy providers, partial streams, and an adapter version
  mismatch.
- The exact deployed revision receives independent code and security review,
  with terminal-green CI and operator recovery evidence before any activation.

## Security consequences

- Subscription OAuth grants are high-value credentials and require the same
  lifecycle controls as service credentials.
- Downstream virtual keys reduce provider-key exposure but do not establish
  user, tenant, agent, or runtime authority.
- Automatic retry or failover can duplicate tool and external side effects
  unless Mosaic owns operation IDs, durable receipts, and reconciliation.
- Gateway telemetry can contain prompts, tool schemas, and model output;
  redaction and retention policy must apply before persistence.
- A localhost translation endpoint must remain loopback-only, process-isolated,
  authenticated where applicable, and outside the Mosaic Gateway ingress path.
- A model-egress gateway outage must not weaken Mosaic authorization, lease
  fencing, tenant isolation, approval, or runtime-session boundaries.

## Acceptance before any production use

The following are minimum exit criteria for a future, separately approved
implementation; they are not satisfied by this RFC:

1. The recorded approval and provider-terms review cover the exact integration.
2. Credential lifecycle, revocation, rotation, redaction, and incident drills
   are documented and exercised.
3. Tenant-bound authorization remains entirely in Mosaic Gateway and passes
   cross-tenant negative tests.
4. `IProviderAdapter` contract tests pass without treating
   `AgentRuntimeProvider` tests as substitutes.
5. Failure injection proves no duplicate side effects across retries or
   provider failover, with durable idempotency receipts.
6. Streaming, tools, cancellation, reasoning policy, errors, and audit
   correlation are qualified for the exact model path.
7. Rollback to the prior provider path is exercised and operator-verifiable.
8. Independent code and security reviews approve the exact deployed revision.

## Follow-up

- #754 owns cross-runtime logical identity, checkpoint, receipt, adapter, and
  failover work.
- #755 / PR #757 implements the first logical identity and connector
  lease/fencing boundary.
- A later issue may prototype LiteLLM or Bifrost behind the model-provider
  boundary only after the recorded approval, security, tenant, idempotency,
  contract, and rollback prerequisites are met.
- The page remains **Draft / proposed** until an explicit decision changes its
  status. Until then, it must not be cited as an approved architecture,
  current integration, or operational runbook.
