docs: migrate optional AI egress RFC

2026-08-10 18:25:38 -05:00
parent fa35c6abed
commit 0692d999f6
2 changed files with 371 additions and 151 deletions
@@ -0,0 +1,371 @@
+# RFC: Optional AI Egress Gateways
+
+> **Status:** Draft / proposed — not approved, not current, and not integrated.
+>
+> **Authority:** Non-operative design proposal. This document does not authorize an
+> integration, deployment, provider selection, credential flow, or production use.
+> It is intentionally an RFC rather than an approved architecture decision.
+>
+> **Scope:** Optional model/inference egress only. `IProviderAdapter` and
+> `AgentRuntimeProvider` are separate boundaries; this RFC does not merge them.
+
+- **Date:** 2026-07-14
+- **Related issues:** #754, #755
+- **Decision owner:** Mosaic Gateway / provider-adapter architecture
+
+## Context
+
+The emergency Mos continuity path kept Claude Code as the harness and translated
+Anthropic Messages traffic to Codex OAuth through a small localhost proxy. That
+preserved the existing Claude Discord plugin and transcript, but exposed two
+architectural facts:
+
+1. Harness identity, channel entitlement, provider credentials, inference
+   transport, and runtime sessions are separate concerns.
+2. A generic AI gateway could eventually improve model routing, budgets, and
+   observability, but it must not become Mosaic's identity, authorization,
+   tenant, connector, or orchestration boundary.
+
+The Tess qualification work also found that provider rebinding is not, by
+itself, identity-continuous failover. Any future design still needs a logical
+agent identity, durable connector lease/fencing, canonical handoff/checkpoint,
+exactly-once receipts, concrete runtime adapters, and cross-runtime rollback
+validation.
+
+This page is a reclassified migration of the historical egress-gateway
+proposal. The move to `docs/DEVELOPER-GUIDE/architecture/rfcs/` does not promote
+it to a decision and does not change runtime behavior.
+
+## Current status and non-goals
+
+This RFC describes a possible future integration boundary. It does **not** say
+that any of the candidates below is supported by Mosaic Gateway today:
+
+- **LiteLLM is not integrated.** It is only a candidate for a future model-egress
+  adapter and remains subject to the prerequisites in this RFC.
+- **Bifrost is not integrated.** Its routing, virtual-key, and failover features
+  are research inputs only, not Mosaic authority.
+- **Claudex and `claude-code-proxy` are experimental harness tooling, not
+  gateway egress.** The `mosaic claudex` path runs a Claude Code harness against
+  a local translation proxy for evaluation. It is not a Mosaic Gateway egress
+  integration, does not define the `IProviderAdapter` model contract, and does
+  not define the `AgentRuntimeProvider` runtime/session contract.
+
+In particular, this RFC does not propose that a generic gateway receive channel
+traffic directly, own Mosaic sessions, replace the runtime-provider registry,
+or become a second authorization or tenant system. No database, service, or
+runtime change is implied by this document.
+
+## Corrected contract boundary
+
+The historical proposal conflated two different adapter families. They must
+remain separate.
+
+### `IProviderAdapter`: model and inference egress
+
+`IProviderAdapter` is the model-provider abstraction. Its current contract
+covers provider registration, model discovery, provider health, and a future
+direct completion stream. `ProviderService` aggregates these adapters and
+connects model availability to the Pi `ModelRegistry`.
+
+It does **not** represent an agent process or session. It does not own channel
+identity, Mosaic actor or tenant identity, logical-agent ownership, runtime
+session IDs, runtime attachments, handoff state, or tool authorization. An
+optional LiteLLM, Bifrost, or similar model gateway would therefore be a
+candidate for this model-egress path only, after approval and qualification.
+
+### `AgentRuntimeProvider`: runtime and session boundary
+
+`AgentRuntimeProvider` is the runtime-neutral session contract. It covers
+runtime capabilities and health plus operations such as listing sessions,
+reading session trees, streaming a session, sending a message, attaching or
+detaching, and terminating a session. Its `RuntimeScope` is derived from
+trusted actor, tenant, channel, and correlation context.
+
+The runtime-provider service performs the gateway-side capability, authorization,
+approval, and metadata-only audit boundary before invoking a runtime provider.
+It is not a model-egress gateway and must not be substituted for
+`IProviderAdapter`. A runtime implementation may use model selection or a model
+service internally, but that dependency is an explicit composition between two
+contracts; it does not make the runtime provider a model provider or make a
+model gateway a session provider.
+
+The checked-in contract and implementation references are:
+
+- [`IProviderAdapter` and provider types](../../../../packages/types/src/provider/index.ts)
+- [`ProviderService`](../../../../apps/gateway/src/agent/provider.service.ts)
+- [`AgentRuntimeProvider`](../../../../packages/types/src/agent/agent-runtime-provider.ts)
+- [`RuntimeProviderService`](../../../../apps/gateway/src/agent/runtime-provider-registry.service.ts)
+- [`AgentRuntimeProviderRegistry`](../../../../packages/agent/src/runtime-provider-registry.ts)
+
+### Proposed relationship
+
+If a future implementation is approved, the model and runtime paths remain
+parallel and explicitly composed at the Mosaic boundary:
+
+```text
+Discord / Matrix / CLI / web
+             |
+             v
+Mosaic Gateway: authenticated actor + tenant, policy, approvals,
+               logical agent, connector lease/fence, audit, checkpoints,
+               idempotency, and side-effect receipts
+             |
+       +-----+-----------------------+
+       |                             |
+       v                             v
+model request path              runtime/session path
+ProviderService / routing      RuntimeProviderService
+       |                             |
+       v                             v
+IProviderAdapter                AgentRuntimeProvider
+       |                             |
+       v                             v
+optional model-egress           native or external runtime/session
+gateway (future only)           transport
+       |
+       v
+upstream model/provider
+```
+
+The optional egress gateway may receive only a validated model request from the
+Mosaic model path. It is not a channel ingress, runtime-session transport,
+connector owner, or source of Mosaic identity.
+
+## Draft proposal
+
+This RFC proposes, for review only, that Mosaic could support an optional
+model-egress gateway through a dedicated `IProviderAdapter` implementation or
+adapter-owned model transport. Any such integration would remain subordinate
+to Mosaic Gateway policy and would not add a second runtime/session boundary.
+
+Mosaic Gateway would remain authoritative for:
+
+- authenticated actor and tenant identity;
+- logical-agent identity, connector binding, lease epoch, and stale-holder
+  fencing;
+- authorization, approval, and model/tool policy;
+- runtime/session access through the separate `AgentRuntimeProvider` boundary;
+- audit correlation, redaction, retention, and operational evidence;
+- canonical handoff, checkpoint, and recovery state; and
+- idempotency keys, operation identity, and durable side-effect receipts.
+
+An optional model-egress gateway must not:
+
+- receive channel ingress directly;
+- authorize tools, connector ownership, runtime sessions, or approvals;
+- define Mosaic actors, tenants, logical agents, or session identity;
+- treat downstream virtual keys as Mosaic principals;
+- persist raw Mosaic handoffs, channel credentials, or unredacted telemetry;
+- bypass adapter capability negotiation or gateway policy;
+- retry or fail over an operation whose side-effect state is ambiguous; or
+- silently select an unhealthy or unauthorized provider merely to return a
+  result.
+
+## Candidate assessment
+
+These dispositions are deliberately non-integrated and do not constitute
+approval.
+
+### LiteLLM
+
+**Disposition:** Not integrated; future candidate for a formal
+`IProviderAdapter` model-egress prototype only.
+
+Potentially useful features include broad provider routing, virtual keys,
+budgets, observability, and OpenAI/Anthropic-compatible surfaces. A future
+review must verify, rather than assume, the exact ChatGPT subscription OAuth
+flow, supported models, provider terms, token storage, encryption, revocation,
+refresh, scope, and incident response.
+
+A prototype would also have to prove streaming, tool calls, reasoning controls,
+cancellation, tenant isolation, audit-correlation preservation, retry behavior,
+and idempotency. Virtual keys must remain downstream credentials and must not
+become Mosaic actors or tenants. Channel ingress and connector credentials
+would remain outside LiteLLM.
+
+Source references:
+
+- [LiteLLM ChatGPT subscription provider](https://docs.litellm.ai/docs/providers/chatgpt)
+- [LiteLLM providers](https://docs.litellm.ai/docs/providers)
+
+### Bifrost
+
+**Disposition:** Not integrated; future candidate for governance and routing
+research, with subscription OAuth compatibility unverified.
+
+Virtual keys, budgets, rate limits, weighted load balancing, and provider
+failover may inform a future Mosaic egress design, but they are not Mosaic
+authority. A future review must verify Codex/ChatGPT subscription OAuth rather
+than assume API-key compatibility, map all policy to server-derived Mosaic
+tenants, prove that failover preserves connector leases and approvals, and
+redact request/response telemetry before persistence.
+
+Automatic failover must be disabled or constrained whenever policy, approval,
+or side-effect state is ambiguous.
+
+Source references:
+
+- [Bifrost overview](https://docs.getbifrost.ai/overview)
+- [Bifrost repository](https://github.com/maximhq/bifrost)
+
+### Claudex / `claude-code-proxy`
+
+**Disposition:** Experimental harness overlay and local translation proxy; not
+Mosaic Gateway egress and not an approved provider integration.
+
+The reviewed path runs GPT models inside the Claude Code harness through a
+local `claude-code-proxy` that translates Anthropic Messages traffic to a
+ChatGPT-subscription (Codex OAuth) backend. It is intended for evaluation and
+must not be described as a current Mosaic model gateway, an `IProviderAdapter`
+implementation, or an `AgentRuntimeProvider` implementation.
+
+Its constraints remain important: it is not a multi-tenant control plane, it
+must not own connector leasing or canonical handoff, and local proxy
+credentials and listener exposure require isolation. The current Mosaic
+launcher documents its experimental status and isolated Claude configuration:
+
+- [`mosaic` claudex documentation](../../../../packages/mosaic/README.md)
+- [`claudex` launch composition](../../../../packages/mosaic/src/commands/claudex.ts)
+- [`claude-code-proxy`](https://github.com/raine/claude-code-proxy)
+
+### `teremterem/claude-code-gpt-5-codex`
+
+**Disposition:** Historical recipe; not selected and not integrated.
+
+The reviewed repository uses `OPENAI_API_KEY`, tells previously authenticated
+Claude users to log out, and documents a Claude Web Search schema
+incompatibility. That does not satisfy the subscription-OAuth plus built-in-
+channel continuity requirement observed in the Mos cutover.
+
+Source references:
+
+- [Repository](https://github.com/teremterem/claude-code-gpt-5-codex)
+- [Environment template](https://github.com/teremterem/claude-code-gpt-5-codex/blob/main/.env.template)
+
+## Prerequisites before approval or integration
+
+No candidate may be integrated, enabled as a Mosaic feature, or treated as a
+current supported path until all of the following are complete for the exact
+provider, adapter, configuration, and deployed revision.
+
+### 1. Recorded approval and governance
+
+- An architecture decision explicitly approves the model-egress scope and
+  records that `IProviderAdapter` remains separate from `AgentRuntimeProvider`.
+- Product/operations ownership, provider terms review, and independent security
+  review are recorded against the exact change and provider revision.
+- The approval identifies allowed providers, models, regions, data handling,
+  rollback owner, support status, and a time-bounded experimental or production
+  phase. This RFC itself is not that approval.
+- No candidate is advertised as integrated, current, or production-ready while
+  the approval record is absent or expired.
+
+### 2. Security and credential controls
+
+- A threat model covers prompt/tool data, model output, streaming, cancellation,
+  retries, failover, SSRF, endpoint authentication, TLS, egress allowlists,
+  supply-chain risk, provider terms, and denial-of-service behavior.
+- OAuth tokens, API keys, virtual keys, refresh tokens, and proxy credentials
+  have documented ownership, storage, encryption, rotation, revocation,
+  expiry, least-privilege scope, and incident-response procedures. Secrets do
+  not enter prompts, logs, traces, audit payloads, or commits.
+- Request, response, tool-schema, and error telemetry is classified and
+  redacted before persistence or external channel delivery. Retention and
+  deletion are tenant-scoped.
+- The adapter fails closed on missing, expired, revoked, malformed, or
+  unauthorized credentials and on uncertain provider health. Loopback-only
+  experimental proxies remain process-isolated and cannot become a gateway
+  bypass.
+
+### 3. Actor and tenant isolation
+
+- Actor and tenant identity are derived from authenticated Mosaic context at
+  the Gateway; callers cannot select a tenant, logical agent, credential, or
+  provider principal by supplying an ID.
+- Downstream virtual keys and provider account identifiers are mapped to
+  server-owned tenant policy. They never grant Mosaic authorization and never
+  replace RBAC, approvals, connector leases, or runtime scope.
+- Model configuration, budgets, rate limits, data residency, credential use,
+  logs, caches, and failure handling are isolated per tenant. Cross-tenant
+  reads, writes, cache hits, telemetry, and failover are denied and tested.
+- The egress adapter receives only the minimum scoped request needed for the
+  authorized model operation; channel ingress and runtime/session control stay
+  in Mosaic-owned boundaries.
+
+### 4. Idempotency and side-effect safety
+
+- Mosaic assigns a durable operation ID and idempotency key before an egress
+  request can cause a tool or external side effect. The provider gateway must
+  preserve the correlation and idempotency metadata or be wrapped by an
+  adapter that does so.
+- Durable receipts record request, attempt, provider, outcome, and replay state
+  without storing unredacted content. Retries and failover are allowed only
+  when the operation contract proves they cannot duplicate a side effect.
+- Ambiguous timeout, disconnect, stream-resume, cancellation, and provider
+  failover outcomes fail closed until the receipt is reconciled. The egress
+  gateway must not claim exactly-once behavior that Mosaic has not proven.
+- Failure injection demonstrates no duplicate tool, connector, channel, or
+  external side effect across gateway, adapter, proxy, and provider retries.
+
+### 5. Contract and rollback qualification
+
+- Separate contract tests pass for `IProviderAdapter` model registration,
+  health, streaming, tools, reasoning controls, cancellation, errors, and
+  audit correlation.
+- Separate `AgentRuntimeProvider` contract tests continue to pass for runtime
+  capability negotiation, session ownership, approval, attach/detach,
+  termination, and normalized events. A model-egress candidate must not be
+  used as evidence for runtime/session compatibility.
+- A verified rollback to the prior model path is exercised, including revoked
+  credentials, unhealthy providers, partial streams, and an adapter version
+  mismatch.
+- The exact deployed revision receives independent code and security review,
+  with terminal-green CI and operator recovery evidence before any activation.
+
+## Security consequences
+
+- Subscription OAuth grants are high-value credentials and require the same
+  lifecycle controls as service credentials.
+- Downstream virtual keys reduce provider-key exposure but do not establish
+  user, tenant, agent, or runtime authority.
+- Automatic retry or failover can duplicate tool and external side effects
+  unless Mosaic owns operation IDs, durable receipts, and reconciliation.
+- Gateway telemetry can contain prompts, tool schemas, and model output;
+  redaction and retention policy must apply before persistence.
+- A localhost translation endpoint must remain loopback-only, process-isolated,
+  authenticated where applicable, and outside the Mosaic Gateway ingress path.
+- A model-egress gateway outage must not weaken Mosaic authorization, lease
+  fencing, tenant isolation, approval, or runtime-session boundaries.
+
+## Acceptance before any production use
+
+The following are minimum exit criteria for a future, separately approved
+implementation; they are not satisfied by this RFC:
+
+1. The recorded approval and provider-terms review cover the exact integration.
+2. Credential lifecycle, revocation, rotation, redaction, and incident drills
+   are documented and exercised.
+3. Tenant-bound authorization remains entirely in Mosaic Gateway and passes
+   cross-tenant negative tests.
+4. `IProviderAdapter` contract tests pass without treating
+   `AgentRuntimeProvider` tests as substitutes.
+5. Failure injection proves no duplicate side effects across retries or
+   provider failover, with durable idempotency receipts.
+6. Streaming, tools, cancellation, reasoning policy, errors, and audit
+   correlation are qualified for the exact model path.
+7. Rollback to the prior provider path is exercised and operator-verifiable.
+8. Independent code and security reviews approve the exact deployed revision.
+
+## Follow-up
+
+- #754 owns cross-runtime logical identity, checkpoint, receipt, adapter, and
+  failover work.
+- #755 / PR #757 implements the first logical identity and connector
+  lease/fencing boundary.
+- A later issue may prototype LiteLLM or Bifrost behind the model-provider
+  boundary only after the recorded approval, security, tenant, idempotency,
+  contract, and rollback prerequisites are met.
+- The page remains **Draft / proposed** until an explicit decision changes its
+  status. Until then, it must not be cited as an approved architecture,
+  current integration, or operational runbook.