From 0692d999f69c7f59baba8f3b4525252213d84812 Mon Sep 17 00:00:00 2001 From: Jason Woltje Date: Mon, 10 Aug 2026 18:23:44 -0500 Subject: [PATCH] docs: migrate optional AI egress RFC --- .../rfcs/optional-ai-egress-gateways.md | 371 ++++++++++++++++++ .../architecture/ADR-MOS-EGRESS-GATEWAYS.md | 151 ------- 2 files changed, 371 insertions(+), 151 deletions(-) create mode 100644 docs/DEVELOPER-GUIDE/architecture/rfcs/optional-ai-egress-gateways.md delete mode 100644 docs/_old_structure/architecture/ADR-MOS-EGRESS-GATEWAYS.md diff --git a/docs/DEVELOPER-GUIDE/architecture/rfcs/optional-ai-egress-gateways.md b/docs/DEVELOPER-GUIDE/architecture/rfcs/optional-ai-egress-gateways.md new file mode 100644 index 00000000..9db1734c --- /dev/null +++ b/docs/DEVELOPER-GUIDE/architecture/rfcs/optional-ai-egress-gateways.md @@ -0,0 +1,371 @@ +# RFC: Optional AI Egress Gateways + +> **Status:** Draft / proposed — not approved, not current, and not integrated. +> +> **Authority:** Non-operative design proposal. This document does not authorize an +> integration, deployment, provider selection, credential flow, or production use. +> It is intentionally an RFC rather than an approved architecture decision. +> +> **Scope:** Optional model/inference egress only. `IProviderAdapter` and +> `AgentRuntimeProvider` are separate boundaries; this RFC does not merge them. + +- **Date:** 2026-07-14 +- **Related issues:** #754, #755 +- **Decision owner:** Mosaic Gateway / provider-adapter architecture + +## Context + +The emergency Mos continuity path kept Claude Code as the harness and translated +Anthropic Messages traffic to Codex OAuth through a small localhost proxy. That +preserved the existing Claude Discord plugin and transcript, but exposed two +architectural facts: + +1. Harness identity, channel entitlement, provider credentials, inference + transport, and runtime sessions are separate concerns. +2. A generic AI gateway could eventually improve model routing, budgets, and + observability, but it must not become Mosaic's identity, authorization, + tenant, connector, or orchestration boundary. + +The Tess qualification work also found that provider rebinding is not, by +itself, identity-continuous failover. Any future design still needs a logical +agent identity, durable connector lease/fencing, canonical handoff/checkpoint, +exactly-once receipts, concrete runtime adapters, and cross-runtime rollback +validation. + +This page is a reclassified migration of the historical egress-gateway +proposal. The move to `docs/DEVELOPER-GUIDE/architecture/rfcs/` does not promote +it to a decision and does not change runtime behavior. + +## Current status and non-goals + +This RFC describes a possible future integration boundary. It does **not** say +that any of the candidates below is supported by Mosaic Gateway today: + +- **LiteLLM is not integrated.** It is only a candidate for a future model-egress + adapter and remains subject to the prerequisites in this RFC. +- **Bifrost is not integrated.** Its routing, virtual-key, and failover features + are research inputs only, not Mosaic authority. +- **Claudex and `claude-code-proxy` are experimental harness tooling, not + gateway egress.** The `mosaic claudex` path runs a Claude Code harness against + a local translation proxy for evaluation. It is not a Mosaic Gateway egress + integration, does not define the `IProviderAdapter` model contract, and does + not define the `AgentRuntimeProvider` runtime/session contract. + +In particular, this RFC does not propose that a generic gateway receive channel +traffic directly, own Mosaic sessions, replace the runtime-provider registry, +or become a second authorization or tenant system. No database, service, or +runtime change is implied by this document. + +## Corrected contract boundary + +The historical proposal conflated two different adapter families. They must +remain separate. + +### `IProviderAdapter`: model and inference egress + +`IProviderAdapter` is the model-provider abstraction. Its current contract +covers provider registration, model discovery, provider health, and a future +direct completion stream. `ProviderService` aggregates these adapters and +connects model availability to the Pi `ModelRegistry`. + +It does **not** represent an agent process or session. It does not own channel +identity, Mosaic actor or tenant identity, logical-agent ownership, runtime +session IDs, runtime attachments, handoff state, or tool authorization. An +optional LiteLLM, Bifrost, or similar model gateway would therefore be a +candidate for this model-egress path only, after approval and qualification. + +### `AgentRuntimeProvider`: runtime and session boundary + +`AgentRuntimeProvider` is the runtime-neutral session contract. It covers +runtime capabilities and health plus operations such as listing sessions, +reading session trees, streaming a session, sending a message, attaching or +detaching, and terminating a session. Its `RuntimeScope` is derived from +trusted actor, tenant, channel, and correlation context. + +The runtime-provider service performs the gateway-side capability, authorization, +approval, and metadata-only audit boundary before invoking a runtime provider. +It is not a model-egress gateway and must not be substituted for +`IProviderAdapter`. A runtime implementation may use model selection or a model +service internally, but that dependency is an explicit composition between two +contracts; it does not make the runtime provider a model provider or make a +model gateway a session provider. + +The checked-in contract and implementation references are: + +- [`IProviderAdapter` and provider types](../../../../packages/types/src/provider/index.ts) +- [`ProviderService`](../../../../apps/gateway/src/agent/provider.service.ts) +- [`AgentRuntimeProvider`](../../../../packages/types/src/agent/agent-runtime-provider.ts) +- [`RuntimeProviderService`](../../../../apps/gateway/src/agent/runtime-provider-registry.service.ts) +- [`AgentRuntimeProviderRegistry`](../../../../packages/agent/src/runtime-provider-registry.ts) + +### Proposed relationship + +If a future implementation is approved, the model and runtime paths remain +parallel and explicitly composed at the Mosaic boundary: + +```text +Discord / Matrix / CLI / web + | + v +Mosaic Gateway: authenticated actor + tenant, policy, approvals, + logical agent, connector lease/fence, audit, checkpoints, + idempotency, and side-effect receipts + | + +-----+-----------------------+ + | | + v v +model request path runtime/session path +ProviderService / routing RuntimeProviderService + | | + v v +IProviderAdapter AgentRuntimeProvider + | | + v v +optional model-egress native or external runtime/session +gateway (future only) transport + | + v +upstream model/provider +``` + +The optional egress gateway may receive only a validated model request from the +Mosaic model path. It is not a channel ingress, runtime-session transport, +connector owner, or source of Mosaic identity. + +## Draft proposal + +This RFC proposes, for review only, that Mosaic could support an optional +model-egress gateway through a dedicated `IProviderAdapter` implementation or +adapter-owned model transport. Any such integration would remain subordinate +to Mosaic Gateway policy and would not add a second runtime/session boundary. + +Mosaic Gateway would remain authoritative for: + +- authenticated actor and tenant identity; +- logical-agent identity, connector binding, lease epoch, and stale-holder + fencing; +- authorization, approval, and model/tool policy; +- runtime/session access through the separate `AgentRuntimeProvider` boundary; +- audit correlation, redaction, retention, and operational evidence; +- canonical handoff, checkpoint, and recovery state; and +- idempotency keys, operation identity, and durable side-effect receipts. + +An optional model-egress gateway must not: + +- receive channel ingress directly; +- authorize tools, connector ownership, runtime sessions, or approvals; +- define Mosaic actors, tenants, logical agents, or session identity; +- treat downstream virtual keys as Mosaic principals; +- persist raw Mosaic handoffs, channel credentials, or unredacted telemetry; +- bypass adapter capability negotiation or gateway policy; +- retry or fail over an operation whose side-effect state is ambiguous; or +- silently select an unhealthy or unauthorized provider merely to return a + result. + +## Candidate assessment + +These dispositions are deliberately non-integrated and do not constitute +approval. + +### LiteLLM + +**Disposition:** Not integrated; future candidate for a formal +`IProviderAdapter` model-egress prototype only. + +Potentially useful features include broad provider routing, virtual keys, +budgets, observability, and OpenAI/Anthropic-compatible surfaces. A future +review must verify, rather than assume, the exact ChatGPT subscription OAuth +flow, supported models, provider terms, token storage, encryption, revocation, +refresh, scope, and incident response. + +A prototype would also have to prove streaming, tool calls, reasoning controls, +cancellation, tenant isolation, audit-correlation preservation, retry behavior, +and idempotency. Virtual keys must remain downstream credentials and must not +become Mosaic actors or tenants. Channel ingress and connector credentials +would remain outside LiteLLM. + +Source references: + +- [LiteLLM ChatGPT subscription provider](https://docs.litellm.ai/docs/providers/chatgpt) +- [LiteLLM providers](https://docs.litellm.ai/docs/providers) + +### Bifrost + +**Disposition:** Not integrated; future candidate for governance and routing +research, with subscription OAuth compatibility unverified. + +Virtual keys, budgets, rate limits, weighted load balancing, and provider +failover may inform a future Mosaic egress design, but they are not Mosaic +authority. A future review must verify Codex/ChatGPT subscription OAuth rather +than assume API-key compatibility, map all policy to server-derived Mosaic +tenants, prove that failover preserves connector leases and approvals, and +redact request/response telemetry before persistence. + +Automatic failover must be disabled or constrained whenever policy, approval, +or side-effect state is ambiguous. + +Source references: + +- [Bifrost overview](https://docs.getbifrost.ai/overview) +- [Bifrost repository](https://github.com/maximhq/bifrost) + +### Claudex / `claude-code-proxy` + +**Disposition:** Experimental harness overlay and local translation proxy; not +Mosaic Gateway egress and not an approved provider integration. + +The reviewed path runs GPT models inside the Claude Code harness through a +local `claude-code-proxy` that translates Anthropic Messages traffic to a +ChatGPT-subscription (Codex OAuth) backend. It is intended for evaluation and +must not be described as a current Mosaic model gateway, an `IProviderAdapter` +implementation, or an `AgentRuntimeProvider` implementation. + +Its constraints remain important: it is not a multi-tenant control plane, it +must not own connector leasing or canonical handoff, and local proxy +credentials and listener exposure require isolation. The current Mosaic +launcher documents its experimental status and isolated Claude configuration: + +- [`mosaic` claudex documentation](../../../../packages/mosaic/README.md) +- [`claudex` launch composition](../../../../packages/mosaic/src/commands/claudex.ts) +- [`claude-code-proxy`](https://github.com/raine/claude-code-proxy) + +### `teremterem/claude-code-gpt-5-codex` + +**Disposition:** Historical recipe; not selected and not integrated. + +The reviewed repository uses `OPENAI_API_KEY`, tells previously authenticated +Claude users to log out, and documents a Claude Web Search schema +incompatibility. That does not satisfy the subscription-OAuth plus built-in- +channel continuity requirement observed in the Mos cutover. + +Source references: + +- [Repository](https://github.com/teremterem/claude-code-gpt-5-codex) +- [Environment template](https://github.com/teremterem/claude-code-gpt-5-codex/blob/main/.env.template) + +## Prerequisites before approval or integration + +No candidate may be integrated, enabled as a Mosaic feature, or treated as a +current supported path until all of the following are complete for the exact +provider, adapter, configuration, and deployed revision. + +### 1. Recorded approval and governance + +- An architecture decision explicitly approves the model-egress scope and + records that `IProviderAdapter` remains separate from `AgentRuntimeProvider`. +- Product/operations ownership, provider terms review, and independent security + review are recorded against the exact change and provider revision. +- The approval identifies allowed providers, models, regions, data handling, + rollback owner, support status, and a time-bounded experimental or production + phase. This RFC itself is not that approval. +- No candidate is advertised as integrated, current, or production-ready while + the approval record is absent or expired. + +### 2. Security and credential controls + +- A threat model covers prompt/tool data, model output, streaming, cancellation, + retries, failover, SSRF, endpoint authentication, TLS, egress allowlists, + supply-chain risk, provider terms, and denial-of-service behavior. +- OAuth tokens, API keys, virtual keys, refresh tokens, and proxy credentials + have documented ownership, storage, encryption, rotation, revocation, + expiry, least-privilege scope, and incident-response procedures. Secrets do + not enter prompts, logs, traces, audit payloads, or commits. +- Request, response, tool-schema, and error telemetry is classified and + redacted before persistence or external channel delivery. Retention and + deletion are tenant-scoped. +- The adapter fails closed on missing, expired, revoked, malformed, or + unauthorized credentials and on uncertain provider health. Loopback-only + experimental proxies remain process-isolated and cannot become a gateway + bypass. + +### 3. Actor and tenant isolation + +- Actor and tenant identity are derived from authenticated Mosaic context at + the Gateway; callers cannot select a tenant, logical agent, credential, or + provider principal by supplying an ID. +- Downstream virtual keys and provider account identifiers are mapped to + server-owned tenant policy. They never grant Mosaic authorization and never + replace RBAC, approvals, connector leases, or runtime scope. +- Model configuration, budgets, rate limits, data residency, credential use, + logs, caches, and failure handling are isolated per tenant. Cross-tenant + reads, writes, cache hits, telemetry, and failover are denied and tested. +- The egress adapter receives only the minimum scoped request needed for the + authorized model operation; channel ingress and runtime/session control stay + in Mosaic-owned boundaries. + +### 4. Idempotency and side-effect safety + +- Mosaic assigns a durable operation ID and idempotency key before an egress + request can cause a tool or external side effect. The provider gateway must + preserve the correlation and idempotency metadata or be wrapped by an + adapter that does so. +- Durable receipts record request, attempt, provider, outcome, and replay state + without storing unredacted content. Retries and failover are allowed only + when the operation contract proves they cannot duplicate a side effect. +- Ambiguous timeout, disconnect, stream-resume, cancellation, and provider + failover outcomes fail closed until the receipt is reconciled. The egress + gateway must not claim exactly-once behavior that Mosaic has not proven. +- Failure injection demonstrates no duplicate tool, connector, channel, or + external side effect across gateway, adapter, proxy, and provider retries. + +### 5. Contract and rollback qualification + +- Separate contract tests pass for `IProviderAdapter` model registration, + health, streaming, tools, reasoning controls, cancellation, errors, and + audit correlation. +- Separate `AgentRuntimeProvider` contract tests continue to pass for runtime + capability negotiation, session ownership, approval, attach/detach, + termination, and normalized events. A model-egress candidate must not be + used as evidence for runtime/session compatibility. +- A verified rollback to the prior model path is exercised, including revoked + credentials, unhealthy providers, partial streams, and an adapter version + mismatch. +- The exact deployed revision receives independent code and security review, + with terminal-green CI and operator recovery evidence before any activation. + +## Security consequences + +- Subscription OAuth grants are high-value credentials and require the same + lifecycle controls as service credentials. +- Downstream virtual keys reduce provider-key exposure but do not establish + user, tenant, agent, or runtime authority. +- Automatic retry or failover can duplicate tool and external side effects + unless Mosaic owns operation IDs, durable receipts, and reconciliation. +- Gateway telemetry can contain prompts, tool schemas, and model output; + redaction and retention policy must apply before persistence. +- A localhost translation endpoint must remain loopback-only, process-isolated, + authenticated where applicable, and outside the Mosaic Gateway ingress path. +- A model-egress gateway outage must not weaken Mosaic authorization, lease + fencing, tenant isolation, approval, or runtime-session boundaries. + +## Acceptance before any production use + +The following are minimum exit criteria for a future, separately approved +implementation; they are not satisfied by this RFC: + +1. The recorded approval and provider-terms review cover the exact integration. +2. Credential lifecycle, revocation, rotation, redaction, and incident drills + are documented and exercised. +3. Tenant-bound authorization remains entirely in Mosaic Gateway and passes + cross-tenant negative tests. +4. `IProviderAdapter` contract tests pass without treating + `AgentRuntimeProvider` tests as substitutes. +5. Failure injection proves no duplicate side effects across retries or + provider failover, with durable idempotency receipts. +6. Streaming, tools, cancellation, reasoning policy, errors, and audit + correlation are qualified for the exact model path. +7. Rollback to the prior provider path is exercised and operator-verifiable. +8. Independent code and security reviews approve the exact deployed revision. + +## Follow-up + +- #754 owns cross-runtime logical identity, checkpoint, receipt, adapter, and + failover work. +- #755 / PR #757 implements the first logical identity and connector + lease/fencing boundary. +- A later issue may prototype LiteLLM or Bifrost behind the model-provider + boundary only after the recorded approval, security, tenant, idempotency, + contract, and rollback prerequisites are met. +- The page remains **Draft / proposed** until an explicit decision changes its + status. Until then, it must not be cited as an approved architecture, + current integration, or operational runbook. diff --git a/docs/_old_structure/architecture/ADR-MOS-EGRESS-GATEWAYS.md b/docs/_old_structure/architecture/ADR-MOS-EGRESS-GATEWAYS.md deleted file mode 100644 index de479f8f..00000000 --- a/docs/_old_structure/architecture/ADR-MOS-EGRESS-GATEWAYS.md +++ /dev/null @@ -1,151 +0,0 @@ -# ADR: Optional AI egress gateways for runtime-neutral Mos - -**Status:** Proposed for controlled prototypes; not approved as Mosaic core - -**Date:** 2026-07-14 - -**Issues:** #754, #755 - -**Decision owner:** Mosaic Gateway / provider-adapter architecture - -## Context - -The emergency Mos continuity path kept Claude Code as the harness and translated Anthropic Messages traffic to Codex OAuth through a small localhost proxy. That preserved the existing Claude Discord plugin and transcript, but exposed two architectural facts: - -1. Harness identity, channel entitlement, provider credentials, and inference transport are separate concerns. -2. A generic AI gateway can improve provider routing, budgets, and observability, but must not become Mosaic's identity, authorization, tenant, or orchestration boundary. - -The Tess qualification report also found that current provider rebinding is not identity-continuous failover. Mosaic still needs a logical agent identity, durable connector lease/fencing, canonical handoff/checkpoint, exactly-once receipts, concrete harness adapters, and cross-harness rollback E2E. - -## Decision - -Mosaic MAY support LiteLLM, Bifrost, the purpose-built Claude/Codex proxy, or future gateways as optional egress implementations behind `IProviderAdapter` / `AgentRuntimeProvider`. - -Mosaic Gateway remains authoritative for: - -- authenticated actor and tenant identity; -- logical agent identity and connector binding; -- authorization, approval, and policy; -- lease epoch and stale-holder fencing; -- audit correlation and redaction; -- canonical handoff/checkpoint state; -- idempotency and side-effect receipts. - -An egress gateway MUST NOT: - -- receive channel ingress directly; -- authorize tools or connector ownership; -- define Mosaic tenant or agent identity; -- persist raw Mosaic handoffs or channel credentials; -- bypass adapter capability negotiation; -- silently fail over when policy, lease, or provider health is uncertain. - -Allowed topology: - -```text -Discord / Matrix / CLI / web - ↓ -Mosaic Gateway: identity, authz, lease/fence, approvals, audit - ↓ -IProviderAdapter / AgentRuntimeProvider - ↓ -optional egress gateway - ↓ -upstream provider or subscription-backed OAuth session -``` - -## Candidate assessment - -### Purpose-built `raine/claude-code-proxy` - -**Disposition:** Approved only for the verified emergency localhost bridge. - -Strengths: - -- explicit Codex device OAuth flow; -- small operational surface; -- Anthropic Messages translation suitable for Claude Code; -- model and reasoning-effort enforcement; -- straightforward loopback systemd supervision and rollback. - -Constraints: - -- not a Mosaic multi-tenant control plane; -- Claude built-in channels still depend on Claude subscription entitlement and feature lookup; -- model aliases can obscure the upstream model unless proxy policy/logs are treated as evidence; -- no replacement for connector leasing, canonical handoff, or exactly-once effects. - -### LiteLLM - -**Disposition:** Candidate for a formal adapter-only prototype and terms/security review. - -Current documentation states that ChatGPT subscription access is available through an OAuth device-code flow. LiteLLM also provides broad provider routing, virtual keys, budgets, observability, and OpenAI/Anthropic-compatible surfaces. - -Required prototype gates: - -- verify the exact ChatGPT subscription OAuth flow and supported models against current provider terms; -- document token location, encryption, revocation, refresh, scope, and incident response; -- prove tenant isolation and prevent virtual keys from becoming Mosaic principals; -- verify streaming, tool calls, reasoning controls, cancellation, and idempotency metadata; -- fail closed instead of selecting an unhealthy provider merely to return a result; -- demonstrate that Mosaic audit correlation survives gateway retries/failover; -- keep channel ingress and connector credentials outside LiteLLM. - -Source references: - -- [LiteLLM ChatGPT subscription provider](https://docs.litellm.ai/docs/providers/chatgpt) -- [LiteLLM providers](https://docs.litellm.ai/docs/providers) - -### Bifrost - -**Disposition:** Candidate for governance/routing research; subscription OAuth compatibility unverified. - -Useful concepts include virtual keys, budgets, rate limits, weighted load balancing, and automatic provider failover. Those features may inform Mosaic egress policy, but Bifrost virtual keys are downstream credentials—not Mosaic actors or tenants. - -Required prototype gates: - -- verify Codex/ChatGPT subscription OAuth rather than assuming API-key compatibility; -- map budgets and virtual keys to server-derived Mosaic tenants without duplicating authority; -- prove failover does not violate connector lease, approval, or exactly-once semantics; -- ensure request/response logs are redacted before persistence; -- disable or constrain automatic failover when policy or side-effect state is ambiguous. - -Source references: - -- [Bifrost overview](https://docs.getbifrost.ai/overview) -- [Bifrost repository](https://github.com/maximhq/bifrost) - -### `teremterem/claude-code-gpt-5-codex` - -**Disposition:** Not selected as the emergency implementation; useful as a historical LiteLLM recipe. - -The reviewed repository uses `OPENAI_API_KEY`, tells previously authenticated Claude users to log out, and documents a Claude Web Search schema incompatibility. Logging Claude out conflicts with the channel-entitlement requirement observed in the live Mos cutover. The repository therefore does not, as provided, satisfy subscription-OAuth plus built-in-channel continuity. - -Source references: - -- [Repository](https://github.com/teremterem/claude-code-gpt-5-codex) -- [Environment template](https://github.com/teremterem/claude-code-gpt-5-codex/blob/main/.env.template) - -## Security consequences - -- Subscription OAuth grants are high-value credentials and require the same lifecycle controls as service credentials. -- Downstream virtual keys reduce provider-key exposure but do not establish user, tenant, or agent authority. -- Automatic retry/failover can duplicate tool or external side effects unless Mosaic owns operation IDs and receipts. -- Gateway telemetry can contain prompts, tool schemas, and model output; redaction and retention policy must apply before persistence. -- A localhost unauthenticated translation endpoint must remain loopback-only and process-isolated. - -## Acceptance before production use - -1. Threat model and provider-terms review approved. -2. Credential lifecycle and revocation drill documented and exercised. -3. Adapter contract tests pass for streaming, tools, cancellation, reasoning policy, errors, and audit correlation. -4. Tenant-bound authorization remains entirely in Mosaic Gateway. -5. Failure injection proves no duplicate side effects across retries or provider failover. -6. Rollback to the prior provider path is exercised. -7. Independent code and security reviews approve the exact deployed revision. - -## Follow-up - -- #754 owns cross-harness logical identity, checkpoint, receipt, adapter, and failover work. -- #755 / PR #757 implements the first logical identity and connector lease/fencing boundary. -- A later issue should prototype LiteLLM and Bifrost behind the provider adapter after #755 is merged and independently qualified.