mosaicstack/stack
1
0
Fork 1
Code Issues 245 Pull Requests 7 Actions Packages Projects Releases 15 Wiki Activity

docs: migrate optional AI egress RFC

Browse Source
This commit is contained in:
Jason Woltje
2026-08-10 18:25:38 -05:00
parent fa35c6abed
commit 0692d999f6
2 changed files with 371 additions and 151 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/DEVELOPER-GUIDE/architecture/rfcs/optional-ai-egress-gateways.md
+371
View File
@@ -0,0 +1,371 @@
# RFC: Optional AI Egress Gateways
> **Status:** Draft / proposed — not approved, not current, and not integrated.
>
> **Authority:** Non-operative design proposal. This document does not authorize an
> integration, deployment, provider selection, credential flow, or production use.
> It is intentionally an RFC rather than an approved architecture decision.
>
> **Scope:** Optional model/inference egress only. `IProviderAdapter` and
> `AgentRuntimeProvider` are separate boundaries; this RFC does not merge them.
- **Date:** 2026-07-14
- **Related issues:** #754, #755
- **Decision owner:** Mosaic Gateway / provider-adapter architecture
## Context
The emergency Mos continuity path kept Claude Code as the harness and translated
Anthropic Messages traffic to Codex OAuth through a small localhost proxy. That
preserved the existing Claude Discord plugin and transcript, but exposed two
architectural facts:
1. Harness identity, channel entitlement, provider credentials, inference
transport, and runtime sessions are separate concerns.
2. A generic AI gateway could eventually improve model routing, budgets, and
observability, but it must not become Mosaic's identity, authorization,
tenant, connector, or orchestration boundary.
The Tess qualification work also found that provider rebinding is not, by
itself, identity-continuous failover. Any future design still needs a logical
agent identity, durable connector lease/fencing, canonical handoff/checkpoint,
exactly-once receipts, concrete runtime adapters, and cross-runtime rollback
validation.
This page is a reclassified migration of the historical egress-gateway
proposal. The move to `docs/DEVELOPER-GUIDE/architecture/rfcs/` does not promote
it to a decision and does not change runtime behavior.
## Current status and non-goals
This RFC describes a possible future integration boundary. It does **not** say
that any of the candidates below is supported by Mosaic Gateway today:
- **LiteLLM is not integrated.** It is only a candidate for a future model-egress
adapter and remains subject to the prerequisites in this RFC.
- **Bifrost is not integrated.** Its routing, virtual-key, and failover features
are research inputs only, not Mosaic authority.
- **Claudex and `claude-code-proxy` are experimental harness tooling, not
gateway egress.** The `mosaic claudex` path runs a Claude Code harness against
a local translation proxy for evaluation. It is not a Mosaic Gateway egress
integration, does not define the `IProviderAdapter` model contract, and does
not define the `AgentRuntimeProvider` runtime/session contract.
In particular, this RFC does not propose that a generic gateway receive channel
traffic directly, own Mosaic sessions, replace the runtime-provider registry,
or become a second authorization or tenant system. No database, service, or
runtime change is implied by this document.
## Corrected contract boundary
The historical proposal conflated two different adapter families. They must
remain separate.
### `IProviderAdapter`: model and inference egress
`IProviderAdapter` is the model-provider abstraction. Its current contract
covers provider registration, model discovery, provider health, and a future
direct completion stream. `ProviderService` aggregates these adapters and
connects model availability to the Pi `ModelRegistry`.
It does **not** represent an agent process or session. It does not own channel
identity, Mosaic actor or tenant identity, logical-agent ownership, runtime
session IDs, runtime attachments, handoff state, or tool authorization. An
optional LiteLLM, Bifrost, or similar model gateway would therefore be a
candidate for this model-egress path only, after approval and qualification.
### `AgentRuntimeProvider`: runtime and session boundary
`AgentRuntimeProvider` is the runtime-neutral session contract. It covers
runtime capabilities and health plus operations such as listing sessions,
reading session trees, streaming a session, sending a message, attaching or
detaching, and terminating a session. Its `RuntimeScope` is derived from
trusted actor, tenant, channel, and correlation context.
The runtime-provider service performs the gateway-side capability, authorization,
approval, and metadata-only audit boundary before invoking a runtime provider.
It is not a model-egress gateway and must not be substituted for
`IProviderAdapter`. A runtime implementation may use model selection or a model
service internally, but that dependency is an explicit composition between two
contracts; it does not make the runtime provider a model provider or make a
model gateway a session provider.
The checked-in contract and implementation references are:
- [`IProviderAdapter` and provider types](../../../../packages/types/src/provider/index.ts)
- [`ProviderService`](../../../../apps/gateway/src/agent/provider.service.ts)
- [`AgentRuntimeProvider`](../../../../packages/types/src/agent/agent-runtime-provider.ts)
- [`RuntimeProviderService`](../../../../apps/gateway/src/agent/runtime-provider-registry.service.ts)
- [`AgentRuntimeProviderRegistry`](../../../../packages/agent/src/runtime-provider-registry.ts)
### Proposed relationship
If a future implementation is approved, the model and runtime paths remain
parallel and explicitly composed at the Mosaic boundary:
```text
Discord / Matrix / CLI / web
|
v
Mosaic Gateway: authenticated actor + tenant, policy, approvals,
logical agent, connector lease/fence, audit, checkpoints,
idempotency, and side-effect receipts
|
+-----+-----------------------+
| |
v v
model request path runtime/session path
ProviderService / routing RuntimeProviderService
| |
v v
IProviderAdapter AgentRuntimeProvider
| |
v v
optional model-egress native or external runtime/session
gateway (future only) transport
|
v
upstream model/provider
```
The optional egress gateway may receive only a validated model request from the
Mosaic model path. It is not a channel ingress, runtime-session transport,
connector owner, or source of Mosaic identity.
## Draft proposal
This RFC proposes, for review only, that Mosaic could support an optional
model-egress gateway through a dedicated `IProviderAdapter` implementation or
adapter-owned model transport. Any such integration would remain subordinate
to Mosaic Gateway policy and would not add a second runtime/session boundary.
Mosaic Gateway would remain authoritative for:
- authenticated actor and tenant identity;
- logical-agent identity, connector binding, lease epoch, and stale-holder
fencing;
- authorization, approval, and model/tool policy;
- runtime/session access through the separate `AgentRuntimeProvider` boundary;
- audit correlation, redaction, retention, and operational evidence;
- canonical handoff, checkpoint, and recovery state; and
- idempotency keys, operation identity, and durable side-effect receipts.
An optional model-egress gateway must not:
- receive channel ingress directly;
- authorize tools, connector ownership, runtime sessions, or approvals;
- define Mosaic actors, tenants, logical agents, or session identity;
- treat downstream virtual keys as Mosaic principals;
- persist raw Mosaic handoffs, channel credentials, or unredacted telemetry;
- bypass adapter capability negotiation or gateway policy;
- retry or fail over an operation whose side-effect state is ambiguous; or
- silently select an unhealthy or unauthorized provider merely to return a
result.
## Candidate assessment
These dispositions are deliberately non-integrated and do not constitute
approval.
### LiteLLM
**Disposition:** Not integrated; future candidate for a formal
`IProviderAdapter` model-egress prototype only.
Potentially useful features include broad provider routing, virtual keys,
budgets, observability, and OpenAI/Anthropic-compatible surfaces. A future
review must verify, rather than assume, the exact ChatGPT subscription OAuth
flow, supported models, provider terms, token storage, encryption, revocation,
refresh, scope, and incident response.
A prototype would also have to prove streaming, tool calls, reasoning controls,
cancellation, tenant isolation, audit-correlation preservation, retry behavior,
and idempotency. Virtual keys must remain downstream credentials and must not
become Mosaic actors or tenants. Channel ingress and connector credentials
would remain outside LiteLLM.
Source references:
- [LiteLLM ChatGPT subscription provider](https://docs.litellm.ai/docs/providers/chatgpt)
- [LiteLLM providers](https://docs.litellm.ai/docs/providers)
### Bifrost
**Disposition:** Not integrated; future candidate for governance and routing
research, with subscription OAuth compatibility unverified.
Virtual keys, budgets, rate limits, weighted load balancing, and provider
failover may inform a future Mosaic egress design, but they are not Mosaic
authority. A future review must verify Codex/ChatGPT subscription OAuth rather
than assume API-key compatibility, map all policy to server-derived Mosaic
tenants, prove that failover preserves connector leases and approvals, and
redact request/response telemetry before persistence.
Automatic failover must be disabled or constrained whenever policy, approval,
or side-effect state is ambiguous.
Source references:
- [Bifrost overview](https://docs.getbifrost.ai/overview)
- [Bifrost repository](https://github.com/maximhq/bifrost)
### Claudex / `claude-code-proxy`
**Disposition:** Experimental harness overlay and local translation proxy; not
Mosaic Gateway egress and not an approved provider integration.
The reviewed path runs GPT models inside the Claude Code harness through a
local `claude-code-proxy` that translates Anthropic Messages traffic to a
ChatGPT-subscription (Codex OAuth) backend. It is intended for evaluation and
must not be described as a current Mosaic model gateway, an `IProviderAdapter`
implementation, or an `AgentRuntimeProvider` implementation.
Its constraints remain important: it is not a multi-tenant control plane, it
must not own connector leasing or canonical handoff, and local proxy
credentials and listener exposure require isolation. The current Mosaic
launcher documents its experimental status and isolated Claude configuration:
- [`mosaic` claudex documentation](../../../../packages/mosaic/README.md)
- [`claudex` launch composition](../../../../packages/mosaic/src/commands/claudex.ts)
- [`claude-code-proxy`](https://github.com/raine/claude-code-proxy)
### `teremterem/claude-code-gpt-5-codex`
**Disposition:** Historical recipe; not selected and not integrated.
The reviewed repository uses `OPENAI_API_KEY`, tells previously authenticated
Claude users to log out, and documents a Claude Web Search schema
incompatibility. That does not satisfy the subscription-OAuth plus built-in-
channel continuity requirement observed in the Mos cutover.
Source references:
- [Repository](https://github.com/teremterem/claude-code-gpt-5-codex)
- [Environment template](https://github.com/teremterem/claude-code-gpt-5-codex/blob/main/.env.template)
## Prerequisites before approval or integration
No candidate may be integrated, enabled as a Mosaic feature, or treated as a
current supported path until all of the following are complete for the exact
provider, adapter, configuration, and deployed revision.
### 1. Recorded approval and governance
- An architecture decision explicitly approves the model-egress scope and
records that `IProviderAdapter` remains separate from `AgentRuntimeProvider`.
- Product/operations ownership, provider terms review, and independent security
review are recorded against the exact change and provider revision.
- The approval identifies allowed providers, models, regions, data handling,
rollback owner, support status, and a time-bounded experimental or production
phase. This RFC itself is not that approval.
- No candidate is advertised as integrated, current, or production-ready while
the approval record is absent or expired.
### 2. Security and credential controls
- A threat model covers prompt/tool data, model output, streaming, cancellation,
retries, failover, SSRF, endpoint authentication, TLS, egress allowlists,
supply-chain risk, provider terms, and denial-of-service behavior.
- OAuth tokens, API keys, virtual keys, refresh tokens, and proxy credentials
have documented ownership, storage, encryption, rotation, revocation,
expiry, least-privilege scope, and incident-response procedures. Secrets do
not enter prompts, logs, traces, audit payloads, or commits.
- Request, response, tool-schema, and error telemetry is classified and
redacted before persistence or external channel delivery. Retention and
deletion are tenant-scoped.
- The adapter fails closed on missing, expired, revoked, malformed, or
unauthorized credentials and on uncertain provider health. Loopback-only
experimental proxies remain process-isolated and cannot become a gateway
bypass.
### 3. Actor and tenant isolation
- Actor and tenant identity are derived from authenticated Mosaic context at
the Gateway; callers cannot select a tenant, logical agent, credential, or
provider principal by supplying an ID.
- Downstream virtual keys and provider account identifiers are mapped to
server-owned tenant policy. They never grant Mosaic authorization and never
replace RBAC, approvals, connector leases, or runtime scope.
- Model configuration, budgets, rate limits, data residency, credential use,
logs, caches, and failure handling are isolated per tenant. Cross-tenant
reads, writes, cache hits, telemetry, and failover are denied and tested.
- The egress adapter receives only the minimum scoped request needed for the
authorized model operation; channel ingress and runtime/session control stay
in Mosaic-owned boundaries.
### 4. Idempotency and side-effect safety
- Mosaic assigns a durable operation ID and idempotency key before an egress
request can cause a tool or external side effect. The provider gateway must
preserve the correlation and idempotency metadata or be wrapped by an
adapter that does so.
- Durable receipts record request, attempt, provider, outcome, and replay state
without storing unredacted content. Retries and failover are allowed only
when the operation contract proves they cannot duplicate a side effect.
- Ambiguous timeout, disconnect, stream-resume, cancellation, and provider
failover outcomes fail closed until the receipt is reconciled. The egress
gateway must not claim exactly-once behavior that Mosaic has not proven.
- Failure injection demonstrates no duplicate tool, connector, channel, or
external side effect across gateway, adapter, proxy, and provider retries.
### 5. Contract and rollback qualification
- Separate contract tests pass for `IProviderAdapter` model registration,
health, streaming, tools, reasoning controls, cancellation, errors, and
audit correlation.
- Separate `AgentRuntimeProvider` contract tests continue to pass for runtime
capability negotiation, session ownership, approval, attach/detach,
termination, and normalized events. A model-egress candidate must not be
used as evidence for runtime/session compatibility.
- A verified rollback to the prior model path is exercised, including revoked
credentials, unhealthy providers, partial streams, and an adapter version
mismatch.
- The exact deployed revision receives independent code and security review,
with terminal-green CI and operator recovery evidence before any activation.
## Security consequences
- Subscription OAuth grants are high-value credentials and require the same
lifecycle controls as service credentials.
- Downstream virtual keys reduce provider-key exposure but do not establish
user, tenant, agent, or runtime authority.
- Automatic retry or failover can duplicate tool and external side effects
unless Mosaic owns operation IDs, durable receipts, and reconciliation.
- Gateway telemetry can contain prompts, tool schemas, and model output;
redaction and retention policy must apply before persistence.
- A localhost translation endpoint must remain loopback-only, process-isolated,
authenticated where applicable, and outside the Mosaic Gateway ingress path.
- A model-egress gateway outage must not weaken Mosaic authorization, lease
fencing, tenant isolation, approval, or runtime-session boundaries.
## Acceptance before any production use
The following are minimum exit criteria for a future, separately approved
implementation; they are not satisfied by this RFC:
1. The recorded approval and provider-terms review cover the exact integration.
2. Credential lifecycle, revocation, rotation, redaction, and incident drills
are documented and exercised.
3. Tenant-bound authorization remains entirely in Mosaic Gateway and passes
cross-tenant negative tests.
4. `IProviderAdapter` contract tests pass without treating
`AgentRuntimeProvider` tests as substitutes.
5. Failure injection proves no duplicate side effects across retries or
provider failover, with durable idempotency receipts.
6. Streaming, tools, cancellation, reasoning policy, errors, and audit
correlation are qualified for the exact model path.
7. Rollback to the prior provider path is exercised and operator-verifiable.
8. Independent code and security reviews approve the exact deployed revision.
## Follow-up
- #754 owns cross-runtime logical identity, checkpoint, receipt, adapter, and
failover work.
- #755 / PR #757 implements the first logical identity and connector
lease/fencing boundary.
- A later issue may prototype LiteLLM or Bifrost behind the model-provider
boundary only after the recorded approval, security, tenant, idempotency,
contract, and rollback prerequisites are met.
- The page remains **Draft / proposed** until an explicit decision changes its
status. Until then, it must not be cited as an approved architecture,
current integration, or operational runbook.
docs/_old_structure/architecture/ADR-MOS-EGRESS-GATEWAYS.md
-151
View File
@@ -1,151 +0,0 @@
# ADR: Optional AI egress gateways for runtime-neutral Mos
**Status:** Proposed for controlled prototypes; not approved as Mosaic core
**Date:** 2026-07-14
**Issues:** #754, #755
**Decision owner:** Mosaic Gateway / provider-adapter architecture
## Context
The emergency Mos continuity path kept Claude Code as the harness and translated Anthropic Messages traffic to Codex OAuth through a small localhost proxy. That preserved the existing Claude Discord plugin and transcript, but exposed two architectural facts:
1. Harness identity, channel entitlement, provider credentials, and inference transport are separate concerns.
2. A generic AI gateway can improve provider routing, budgets, and observability, but must not become Mosaic's identity, authorization, tenant, or orchestration boundary.
The Tess qualification report also found that current provider rebinding is not identity-continuous failover. Mosaic still needs a logical agent identity, durable connector lease/fencing, canonical handoff/checkpoint, exactly-once receipts, concrete harness adapters, and cross-harness rollback E2E.
## Decision
Mosaic MAY support LiteLLM, Bifrost, the purpose-built Claude/Codex proxy, or future gateways as optional egress implementations behind `IProviderAdapter` / `AgentRuntimeProvider`.
Mosaic Gateway remains authoritative for:
- authenticated actor and tenant identity;
- logical agent identity and connector binding;
- authorization, approval, and policy;
- lease epoch and stale-holder fencing;
- audit correlation and redaction;
- canonical handoff/checkpoint state;
- idempotency and side-effect receipts.
An egress gateway MUST NOT:
- receive channel ingress directly;
- authorize tools or connector ownership;
- define Mosaic tenant or agent identity;
- persist raw Mosaic handoffs or channel credentials;
- bypass adapter capability negotiation;
- silently fail over when policy, lease, or provider health is uncertain.
Allowed topology:
```text
Discord / Matrix / CLI / web
↓
Mosaic Gateway: identity, authz, lease/fence, approvals, audit
↓
IProviderAdapter / AgentRuntimeProvider
↓
optional egress gateway
↓
upstream provider or subscription-backed OAuth session
```
## Candidate assessment
### Purpose-built `raine/claude-code-proxy`
**Disposition:** Approved only for the verified emergency localhost bridge.
Strengths:
- explicit Codex device OAuth flow;
- small operational surface;
- Anthropic Messages translation suitable for Claude Code;
- model and reasoning-effort enforcement;
- straightforward loopback systemd supervision and rollback.
Constraints:
- not a Mosaic multi-tenant control plane;
- Claude built-in channels still depend on Claude subscription entitlement and feature lookup;
- model aliases can obscure the upstream model unless proxy policy/logs are treated as evidence;
- no replacement for connector leasing, canonical handoff, or exactly-once effects.
### LiteLLM
**Disposition:** Candidate for a formal adapter-only prototype and terms/security review.
Current documentation states that ChatGPT subscription access is available through an OAuth device-code flow. LiteLLM also provides broad provider routing, virtual keys, budgets, observability, and OpenAI/Anthropic-compatible surfaces.
Required prototype gates:
- verify the exact ChatGPT subscription OAuth flow and supported models against current provider terms;
- document token location, encryption, revocation, refresh, scope, and incident response;
- prove tenant isolation and prevent virtual keys from becoming Mosaic principals;
- verify streaming, tool calls, reasoning controls, cancellation, and idempotency metadata;
- fail closed instead of selecting an unhealthy provider merely to return a result;
- demonstrate that Mosaic audit correlation survives gateway retries/failover;
- keep channel ingress and connector credentials outside LiteLLM.
Source references:
- [LiteLLM ChatGPT subscription provider](https://docs.litellm.ai/docs/providers/chatgpt)
- [LiteLLM providers](https://docs.litellm.ai/docs/providers)
### Bifrost
**Disposition:** Candidate for governance/routing research; subscription OAuth compatibility unverified.
Useful concepts include virtual keys, budgets, rate limits, weighted load balancing, and automatic provider failover. Those features may inform Mosaic egress policy, but Bifrost virtual keys are downstream credentials—not Mosaic actors or tenants.
Required prototype gates:
- verify Codex/ChatGPT subscription OAuth rather than assuming API-key compatibility;
- map budgets and virtual keys to server-derived Mosaic tenants without duplicating authority;
- prove failover does not violate connector lease, approval, or exactly-once semantics;
- ensure request/response logs are redacted before persistence;
- disable or constrain automatic failover when policy or side-effect state is ambiguous.
Source references:
- [Bifrost overview](https://docs.getbifrost.ai/overview)
- [Bifrost repository](https://github.com/maximhq/bifrost)
### `teremterem/claude-code-gpt-5-codex`
**Disposition:** Not selected as the emergency implementation; useful as a historical LiteLLM recipe.
The reviewed repository uses `OPENAI_API_KEY`, tells previously authenticated Claude users to log out, and documents a Claude Web Search schema incompatibility. Logging Claude out conflicts with the channel-entitlement requirement observed in the live Mos cutover. The repository therefore does not, as provided, satisfy subscription-OAuth plus built-in-channel continuity.
Source references:
- [Repository](https://github.com/teremterem/claude-code-gpt-5-codex)
- [Environment template](https://github.com/teremterem/claude-code-gpt-5-codex/blob/main/.env.template)
## Security consequences
- Subscription OAuth grants are high-value credentials and require the same lifecycle controls as service credentials.
- Downstream virtual keys reduce provider-key exposure but do not establish user, tenant, or agent authority.
- Automatic retry/failover can duplicate tool or external side effects unless Mosaic owns operation IDs and receipts.
- Gateway telemetry can contain prompts, tool schemas, and model output; redaction and retention policy must apply before persistence.
- A localhost unauthenticated translation endpoint must remain loopback-only and process-isolated.
## Acceptance before production use
1. Threat model and provider-terms review approved.
2. Credential lifecycle and revocation drill documented and exercised.
3. Adapter contract tests pass for streaming, tools, cancellation, reasoning policy, errors, and audit correlation.
4. Tenant-bound authorization remains entirely in Mosaic Gateway.
5. Failure injection proves no duplicate side effects across retries or provider failover.
6. Rollback to the prior provider path is exercised.
7. Independent code and security reviews approve the exact deployed revision.
## Follow-up
- #754 owns cross-harness logical identity, checkpoint, receipt, adapter, and failover work.
- #755 / PR #757 implements the first logical identity and connector lease/fencing boundary.
- A later issue should prototype LiteLLM and Bifrost behind the provider adapter after #755 is merged and independently qualified.