Move old docs into _old_structure dir. Create blank GETTING_STARTED.md document
This commit is contained in:
Jason Woltje
@@ -0,0 +1,151 @@
|
||||
# ADR: Optional AI egress gateways for runtime-neutral Mos
|
||||
|
||||
**Status:** Proposed for controlled prototypes; not approved as Mosaic core
|
||||
|
||||
**Date:** 2026-07-14
|
||||
|
||||
**Issues:** #754, #755
|
||||
|
||||
**Decision owner:** Mosaic Gateway / provider-adapter architecture
|
||||
|
||||
## Context
|
||||
|
||||
The emergency Mos continuity path kept Claude Code as the harness and translated Anthropic Messages traffic to Codex OAuth through a small localhost proxy. That preserved the existing Claude Discord plugin and transcript, but exposed two architectural facts:
|
||||
|
||||
1. Harness identity, channel entitlement, provider credentials, and inference transport are separate concerns.
|
||||
2. A generic AI gateway can improve provider routing, budgets, and observability, but must not become Mosaic's identity, authorization, tenant, or orchestration boundary.
|
||||
|
||||
The Tess qualification report also found that current provider rebinding is not identity-continuous failover. Mosaic still needs a logical agent identity, durable connector lease/fencing, canonical handoff/checkpoint, exactly-once receipts, concrete harness adapters, and cross-harness rollback E2E.
|
||||
|
||||
## Decision
|
||||
|
||||
Mosaic MAY support LiteLLM, Bifrost, the purpose-built Claude/Codex proxy, or future gateways as optional egress implementations behind `IProviderAdapter` / `AgentRuntimeProvider`.
|
||||
|
||||
Mosaic Gateway remains authoritative for:
|
||||
|
||||
- authenticated actor and tenant identity;
|
||||
- logical agent identity and connector binding;
|
||||
- authorization, approval, and policy;
|
||||
- lease epoch and stale-holder fencing;
|
||||
- audit correlation and redaction;
|
||||
- canonical handoff/checkpoint state;
|
||||
- idempotency and side-effect receipts.
|
||||
|
||||
An egress gateway MUST NOT:
|
||||
|
||||
- receive channel ingress directly;
|
||||
- authorize tools or connector ownership;
|
||||
- define Mosaic tenant or agent identity;
|
||||
- persist raw Mosaic handoffs or channel credentials;
|
||||
- bypass adapter capability negotiation;
|
||||
- silently fail over when policy, lease, or provider health is uncertain.
|
||||
|
||||
Allowed topology:
|
||||
|
||||
```text
|
||||
Discord / Matrix / CLI / web
|
||||
↓
|
||||
Mosaic Gateway: identity, authz, lease/fence, approvals, audit
|
||||
↓
|
||||
IProviderAdapter / AgentRuntimeProvider
|
||||
↓
|
||||
optional egress gateway
|
||||
↓
|
||||
upstream provider or subscription-backed OAuth session
|
||||
```
|
||||
|
||||
## Candidate assessment
|
||||
|
||||
### Purpose-built `raine/claude-code-proxy`
|
||||
|
||||
**Disposition:** Approved only for the verified emergency localhost bridge.
|
||||
|
||||
Strengths:
|
||||
|
||||
- explicit Codex device OAuth flow;
|
||||
- small operational surface;
|
||||
- Anthropic Messages translation suitable for Claude Code;
|
||||
- model and reasoning-effort enforcement;
|
||||
- straightforward loopback systemd supervision and rollback.
|
||||
|
||||
Constraints:
|
||||
|
||||
- not a Mosaic multi-tenant control plane;
|
||||
- Claude built-in channels still depend on Claude subscription entitlement and feature lookup;
|
||||
- model aliases can obscure the upstream model unless proxy policy/logs are treated as evidence;
|
||||
- no replacement for connector leasing, canonical handoff, or exactly-once effects.
|
||||
|
||||
### LiteLLM
|
||||
|
||||
**Disposition:** Candidate for a formal adapter-only prototype and terms/security review.
|
||||
|
||||
Current documentation states that ChatGPT subscription access is available through an OAuth device-code flow. LiteLLM also provides broad provider routing, virtual keys, budgets, observability, and OpenAI/Anthropic-compatible surfaces.
|
||||
|
||||
Required prototype gates:
|
||||
|
||||
- verify the exact ChatGPT subscription OAuth flow and supported models against current provider terms;
|
||||
- document token location, encryption, revocation, refresh, scope, and incident response;
|
||||
- prove tenant isolation and prevent virtual keys from becoming Mosaic principals;
|
||||
- verify streaming, tool calls, reasoning controls, cancellation, and idempotency metadata;
|
||||
- fail closed instead of selecting an unhealthy provider merely to return a result;
|
||||
- demonstrate that Mosaic audit correlation survives gateway retries/failover;
|
||||
- keep channel ingress and connector credentials outside LiteLLM.
|
||||
|
||||
Source references:
|
||||
|
||||
- [LiteLLM ChatGPT subscription provider](https://docs.litellm.ai/docs/providers/chatgpt)
|
||||
- [LiteLLM providers](https://docs.litellm.ai/docs/providers)
|
||||
|
||||
### Bifrost
|
||||
|
||||
**Disposition:** Candidate for governance/routing research; subscription OAuth compatibility unverified.
|
||||
|
||||
Useful concepts include virtual keys, budgets, rate limits, weighted load balancing, and automatic provider failover. Those features may inform Mosaic egress policy, but Bifrost virtual keys are downstream credentials—not Mosaic actors or tenants.
|
||||
|
||||
Required prototype gates:
|
||||
|
||||
- verify Codex/ChatGPT subscription OAuth rather than assuming API-key compatibility;
|
||||
- map budgets and virtual keys to server-derived Mosaic tenants without duplicating authority;
|
||||
- prove failover does not violate connector lease, approval, or exactly-once semantics;
|
||||
- ensure request/response logs are redacted before persistence;
|
||||
- disable or constrain automatic failover when policy or side-effect state is ambiguous.
|
||||
|
||||
Source references:
|
||||
|
||||
- [Bifrost overview](https://docs.getbifrost.ai/overview)
|
||||
- [Bifrost repository](https://github.com/maximhq/bifrost)
|
||||
|
||||
### `teremterem/claude-code-gpt-5-codex`
|
||||
|
||||
**Disposition:** Not selected as the emergency implementation; useful as a historical LiteLLM recipe.
|
||||
|
||||
The reviewed repository uses `OPENAI_API_KEY`, tells previously authenticated Claude users to log out, and documents a Claude Web Search schema incompatibility. Logging Claude out conflicts with the channel-entitlement requirement observed in the live Mos cutover. The repository therefore does not, as provided, satisfy subscription-OAuth plus built-in-channel continuity.
|
||||
|
||||
Source references:
|
||||
|
||||
- [Repository](https://github.com/teremterem/claude-code-gpt-5-codex)
|
||||
- [Environment template](https://github.com/teremterem/claude-code-gpt-5-codex/blob/main/.env.template)
|
||||
|
||||
## Security consequences
|
||||
|
||||
- Subscription OAuth grants are high-value credentials and require the same lifecycle controls as service credentials.
|
||||
- Downstream virtual keys reduce provider-key exposure but do not establish user, tenant, or agent authority.
|
||||
- Automatic retry/failover can duplicate tool or external side effects unless Mosaic owns operation IDs and receipts.
|
||||
- Gateway telemetry can contain prompts, tool schemas, and model output; redaction and retention policy must apply before persistence.
|
||||
- A localhost unauthenticated translation endpoint must remain loopback-only and process-isolated.
|
||||
|
||||
## Acceptance before production use
|
||||
|
||||
1. Threat model and provider-terms review approved.
|
||||
2. Credential lifecycle and revocation drill documented and exercised.
|
||||
3. Adapter contract tests pass for streaming, tools, cancellation, reasoning policy, errors, and audit correlation.
|
||||
4. Tenant-bound authorization remains entirely in Mosaic Gateway.
|
||||
5. Failure injection proves no duplicate side effects across retries or provider failover.
|
||||
6. Rollback to the prior provider path is exercised.
|
||||
7. Independent code and security reviews approve the exact deployed revision.
|
||||
|
||||
## Follow-up
|
||||
|
||||
- #754 owns cross-harness logical identity, checkpoint, receipt, adapter, and failover work.
|
||||
- #755 / PR #757 implements the first logical identity and connector lease/fencing boundary.
|
||||
- A later issue should prototype LiteLLM and Bifrost behind the provider adapter after #755 is merged and independently qualified.
|
||||
@@ -0,0 +1,751 @@
|
||||
# Channel Protocol Architecture
|
||||
|
||||
**Status:** Official adapter baseline implemented by #756; extended registry/multiplexing remains iterative
|
||||
**Authors:** Mosaic Core Team
|
||||
**Last Updated:** 2026-07-14
|
||||
**Covers:** M7-001 (OfficialChannelAdapter interface), M7-002 (ChannelMessageDto protocol), M7-003 (Matrix integration design), M7-004 (conversation multiplexing), M7-005 (remote auth bridging), M7-006 (agent-to-agent communication via Matrix), M7-007 (multi-user isolation in Matrix)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The channel protocol defines a unified abstraction layer between Mosaic's core messaging infrastructure and the external communication channels it supports (Matrix, Discord, Telegram, TUI, WebUI, and future channels).
|
||||
|
||||
The implemented baseline is exported from `@mosaicstack/types` and consists of four contract groups:
|
||||
|
||||
1. `OfficialChannelAdapter` — transport lifecycle and connection health.
|
||||
2. `ChannelMessageDto` / `ChannelAttachmentDto` — canonical transport data.
|
||||
3. `ChannelConversationRouteDto` — stable logical-agent conversation and authorization address.
|
||||
4. `ChannelResponseTargetDto` — channel/thread destination for replies.
|
||||
|
||||
All channel-specific translation logic lives inside the adapter implementation. Runtime selection does not: gateway durable-session and provider services may rebind the logical session from Claude to Codex, Pi, OpenCode, or another harness without reconnecting the channel adapter.
|
||||
|
||||
---
|
||||
|
||||
## M7-001: OfficialChannelAdapter Interface
|
||||
|
||||
```typescript
|
||||
interface OfficialChannelAdapter {
|
||||
/** Stable, lowercase adapter identifier such as "discord" or "matrix". */
|
||||
readonly name: string;
|
||||
/** Establish both native-channel and gateway connections. */
|
||||
start(): Promise<void>;
|
||||
/** Gracefully close connections and release resources. */
|
||||
stop(): Promise<void>;
|
||||
/** Best-effort health; ordinary disconnection is a result, not an exception. */
|
||||
health(): Promise<{
|
||||
status: 'connected' | 'degraded' | 'disconnected';
|
||||
detail?: string;
|
||||
}>;
|
||||
}
|
||||
```
|
||||
|
||||
The small lifecycle seam lets the gateway host official plugins uniformly without moving native message translation into gateway core. Message ingress remains adapter-owned; gateway policy, durable session routing, auditing, and runtime/provider selection remain gateway-owned.
|
||||
|
||||
### Stable conversation route
|
||||
|
||||
```typescript
|
||||
interface ChannelConversationRouteDto {
|
||||
bindingId: string;
|
||||
logicalAgentId: string;
|
||||
conversationId: string;
|
||||
channelName: string;
|
||||
authorizationChannelId: string;
|
||||
responseTarget: { channelId: string; threadId?: string };
|
||||
}
|
||||
```
|
||||
|
||||
Harness, provider, model, process, and native runtime-session identifiers are forbidden from this route. Runtime adapters consume the gateway's durable logical-session binding; channel adapters consume only the stable route and response target.
|
||||
|
||||
### Typed ingress and egress ports
|
||||
|
||||
`ChannelIngressPort` is the transport-neutral direct-integration seam for official adapters. The current deployed Discord adapter preserves its existing HMAC-signed Socket.IO compatibility ingress so gateway-side service authentication, replay protection, approval handling, and correlation semantics remain unchanged; it normalizes the same `ChannelIngressDto` before signing. The adapter uses a supplied `ChannelIngressPort` directly when a future gateway registration provides one. New adapters must use the shared ports rather than adding channel branches to gateway core.
|
||||
|
||||
`ChannelBindingDto` contains the configuration-owned workspace/channel→logical-agent mapping and paired external principals; credentials are absent. After native allowlist, pairing, and role checks pass, an adapter submits `ChannelIngressDto` to `ChannelIngressPort.receive()`. It includes the normalized message, `ChannelAuthorizedPrincipalDto`, operation, correlation ID, native message ID, and stable route. Unauthorized input never reaches the port.
|
||||
|
||||
Gateway policy and runtime routing produce `ChannelEgressDto`, which `ChannelEgressPort.send()` delivers to the route's response target. Discord's existing HMAC envelope is its authenticated wire encoding of this boundary; future Matrix/Slack adapters use their native authenticated transports while preserving the same actor/operation/correlation semantics.
|
||||
|
||||
### Adapter Registration
|
||||
|
||||
Adapters are registered with the gateway plugin host at startup. The host calls `start()`/`stop()` and may monitor `health()` on a configurable interval. A richer dynamic `ChannelRegistry` remains a compatible future extension of this lifecycle contract.
|
||||
|
||||
```
|
||||
ChannelRegistry
|
||||
└── register(adapter: OfficialChannelAdapter): void
|
||||
└── getAdapter(name: string): OfficialChannelAdapter | null
|
||||
└── listAdapters(): OfficialChannelAdapter[]
|
||||
└── healthAll(): Promise<Record<string, AdapterHealth>>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## M7-002: ChannelMessageDto Protocol
|
||||
|
||||
### Canonical Message Format
|
||||
|
||||
```typescript
|
||||
interface ChannelMessageDto {
|
||||
/**
|
||||
* Globally unique message ID.
|
||||
* Format: UUID v4. Generated by the adapter when receiving, or by Mosaic
|
||||
* when sending. Channel-native IDs are stored in metadata.channelMessageId.
|
||||
*/
|
||||
id: string;
|
||||
|
||||
/**
|
||||
* Channel-native room/conversation/channel identifier.
|
||||
* The adapter populates this from the inbound message.
|
||||
* For outbound messages, the caller supplies the target channel.
|
||||
*/
|
||||
channelName: string;
|
||||
channelId: string;
|
||||
|
||||
/**
|
||||
* Channel-native identifier of the message sender.
|
||||
* For Mosaic-originated messages this is the Mosaic userId or agentId.
|
||||
*/
|
||||
senderId: string;
|
||||
|
||||
/** Sender classification. */
|
||||
senderKind: 'user' | 'agent' | 'system';
|
||||
|
||||
/**
|
||||
* Textual content of the message.
|
||||
* For non-text content types (image, file) this may be an empty string
|
||||
* or an alt-text description; the actual payload is in `attachments`.
|
||||
*/
|
||||
content: string;
|
||||
|
||||
/**
|
||||
* Hint for how `content` should be interpreted and rendered.
|
||||
* - "text" — plain text, no special rendering
|
||||
* - "markdown" — CommonMark markdown
|
||||
* - "code" — code block (use metadata.language for the language tag)
|
||||
* - "image" — binary image; content is empty, see attachments
|
||||
* - "file" — binary file; content is empty, see attachments
|
||||
*/
|
||||
contentKind: 'text' | 'markdown' | 'code' | 'image' | 'file';
|
||||
|
||||
/**
|
||||
* Arbitrary key-value metadata for channel-specific extension fields.
|
||||
* Examples: { channelMessageId, language, reactionEmoji, channelType }.
|
||||
* Adapters should store channel-native IDs here so round-trip correlation
|
||||
* is possible without altering the canonical fields.
|
||||
*/
|
||||
metadata: Readonly<Record<string, ChannelMetadataValue>>;
|
||||
|
||||
/**
|
||||
* Optional thread or reply-chain identifier.
|
||||
* For threaded channels (Matrix, Discord threads, Telegram topics) this
|
||||
* groups messages into a logical thread scoped to the same channelId.
|
||||
*/
|
||||
threadId?: string;
|
||||
|
||||
/**
|
||||
* The canonical message ID this message is a reply to.
|
||||
* Maps to channel-native reply/quote mechanisms in each adapter.
|
||||
*/
|
||||
replyToId?: string;
|
||||
|
||||
/**
|
||||
* Binary or URI-referenced attachments.
|
||||
* Each attachment carries its MIME type and a URL or base64 payload.
|
||||
*/
|
||||
attachments?: readonly ChannelAttachmentDto[];
|
||||
|
||||
/** ISO-8601 wall-clock timestamp when the message was sent/received. */
|
||||
timestamp: string;
|
||||
}
|
||||
|
||||
interface ChannelAttachmentDto {
|
||||
/** Channel-native attachment identifier. */
|
||||
id: string;
|
||||
|
||||
/** Filename or display name. */
|
||||
name: string;
|
||||
|
||||
/** MIME type when supplied by the channel. */
|
||||
mimeType: string | null;
|
||||
|
||||
/**
|
||||
* URL pointing to the attachment, OR a `data:` URI with base64 payload.
|
||||
* Adapters that receive file uploads SHOULD store to object storage and
|
||||
* populate a stable URL here rather than embedding the raw bytes.
|
||||
*/
|
||||
url: string;
|
||||
|
||||
/** Size in bytes, if known. */
|
||||
sizeBytes?: number;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Channel Translation Reference
|
||||
|
||||
The following sections document how each supported channel maps its native message format to and from `ChannelMessageDto`.
|
||||
|
||||
### Matrix
|
||||
|
||||
| ChannelMessageDto field | Matrix equivalent |
|
||||
| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `id` | Generated UUID; `metadata.channelMessageId` = Matrix event ID (`$...`) |
|
||||
| `channelId` | Matrix room ID (`!roomid:homeserver`) |
|
||||
| `senderId` | Matrix user ID (`@user:homeserver`) |
|
||||
| `senderKind` | Always `"user"` for inbound; `"agent"` or `"system"` for outbound |
|
||||
| `content` | `event.content.body` |
|
||||
| `contentKind` | `"markdown"` if `msgtype = m.text` and body contains markdown; `"text"` otherwise; `"image"` for `m.image`; `"file"` for `m.file` |
|
||||
| `threadId` | `event.content['m.relates_to']['event_id']` when `rel_type = m.thread` |
|
||||
| `replyToId` | Mosaic ID looked up from `event.content['m.relates_to']['m.in_reply_to']['event_id']` |
|
||||
| `attachments` | Populated from `url` in `m.image` / `m.file` events |
|
||||
| `timestamp` | `new Date(event.origin_server_ts)` |
|
||||
| `metadata` | `{ channelMessageId, roomId, eventType, unsigned }` |
|
||||
|
||||
**Outbound:** Adapter sends `m.room.message` with `msgtype = m.text` (or `m.notice` for system messages). Markdown content is sent with `format = org.matrix.custom.html` and a rendered HTML body.
|
||||
|
||||
---
|
||||
|
||||
### Discord
|
||||
|
||||
| ChannelMessageDto field | Discord equivalent |
|
||||
| ----------------------- | ----------------------------------------------------------------------- |
|
||||
| `id` | Generated UUID; `metadata.channelMessageId` = Discord message snowflake |
|
||||
| `channelId` | Discord channel ID (snowflake string) |
|
||||
| `senderId` | Discord user ID (snowflake) |
|
||||
| `senderKind` | `"user"` for human members; `"agent"` for bot messages |
|
||||
| `content` | `message.content` |
|
||||
| `contentKind` | `"markdown"` (Discord uses a markdown-like syntax natively) |
|
||||
| `threadId` | `message.thread.id` when the message is inside a thread channel |
|
||||
| `replyToId` | Mosaic ID looked up from `message.referenced_message.id` |
|
||||
| `attachments` | `message.attachments` mapped to `ChannelAttachmentDto` |
|
||||
| `timestamp` | `new Date(message.timestamp)` |
|
||||
| `metadata` | `{ channelMessageId, guildId, channelType, mentions, embeds }` |
|
||||
|
||||
**Outbound:** Adapter calls Discord REST `POST /channels/{id}/messages`. Markdown content is sent as-is (Discord renders it). For `contentKind = "code"` the adapter wraps in triple-backtick fences with the `metadata.language` tag.
|
||||
|
||||
### Discord routing and thread policy
|
||||
|
||||
A configured Discord binding maps `(guildId, parentChannelId)` to a stable logical agent and a trusted gateway agent-config ID. Gateway verifies that configuration's name matches the binding logical agent before session creation. The stable conversation handle is derived from logical agent plus response channel/thread and never includes the active harness, provider, model, process, or agent-config ID.
|
||||
|
||||
| Inbound location/trigger | Conversation and response target |
|
||||
| ------------------------------------------ | --------------------------------------------------------------- |
|
||||
| Authorized untagged parent-channel message | Parent channel; response is sent in-channel |
|
||||
| Authorized bot mention in parent channel | Thread already attached to that message, or a new public thread |
|
||||
| Authorized message already in a thread | Existing thread; no repeated mention and no nested thread |
|
||||
| `/approve` or `/stop <approval>` | Current parent/thread durable session; no new topic is created |
|
||||
|
||||
Authorization order is fixed: guild allowlist → parent-channel allowlist → user allowlist → configured binding/pairing → operation role → per-user/channel message and thread rate limits → thread creation/dispatch. A normal Discord channel's category parent is never treated as the thread authorization parent. If requested thread creation fails, dispatch does not occur because the adapter cannot honor the response target.
|
||||
|
||||
### Discord service ingress security
|
||||
|
||||
The Discord adapter is an authenticated gateway service, not an anonymous Socket.IO client. It presents `DISCORD_SERVICE_TOKEN` during its `/chat` connection and signs each inbound envelope using HMAC-SHA-256. The envelope contains the Discord native message ID and a generated correlation ID. Gateway verifies the service credential, signature, and configured guild/channel/user allowlists before agent dispatch, then rejects duplicate native message IDs inside its bounded replay window. All three allowlists are default-deny and required when the Discord plugin is enabled. The service credential is injected at runtime and is never logged or included in protocol payloads.
|
||||
|
||||
---
|
||||
|
||||
### Telegram
|
||||
|
||||
| ChannelMessageDto field | Telegram equivalent |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------------------- |
|
||||
| `id` | Generated UUID; `metadata.channelMessageId` = Telegram `message_id` (integer) |
|
||||
| `channelId` | Telegram `chat_id` (integer as string) |
|
||||
| `senderId` | Telegram `from.id` (integer as string) |
|
||||
| `senderKind` | `"user"` for human senders; `"agent"` for bot-originated messages |
|
||||
| `content` | `message.text` or `message.caption` |
|
||||
| `contentKind` | `"text"` for plain; `"markdown"` if `parse_mode = MarkdownV2`; `"image"` for `photo`; `"file"` for `document` |
|
||||
| `threadId` | `message.message_thread_id` (for supergroup topics) |
|
||||
| `replyToId` | Mosaic ID looked up from `message.reply_to_message.message_id` |
|
||||
| `attachments` | `photo`, `document`, `video` fields mapped to `ChannelAttachmentDto` |
|
||||
| `timestamp` | `new Date(message.date * 1000)` |
|
||||
| `metadata` | `{ channelMessageId, chatType, fromUsername, forwardFrom }` |
|
||||
|
||||
**Outbound:** Adapter calls Telegram Bot API `sendMessage` with `parse_mode = MarkdownV2` for markdown content. For `contentKind = "image"` or `"file"` it uses `sendPhoto` / `sendDocument`.
|
||||
|
||||
---
|
||||
|
||||
### TUI (Terminal UI)
|
||||
|
||||
The TUI adapter bridges Mosaic's terminal interface (`packages/cli`) to the channel protocol so that TUI sessions can be treated as a first-class channel.
|
||||
|
||||
| ChannelMessageDto field | TUI equivalent |
|
||||
| ----------------------- | ------------------------------------------------------------------ |
|
||||
| `id` | Generated UUID (TUI has no native message IDs) |
|
||||
| `channelId` | `"tui:<conversationId>"` — the active conversation ID |
|
||||
| `senderId` | Authenticated Mosaic `userId` |
|
||||
| `senderKind` | `"user"` for human input; `"agent"` for agent replies |
|
||||
| `content` | Raw text from stdin / agent output |
|
||||
| `contentKind` | `"text"` for input; `"markdown"` for agent responses |
|
||||
| `threadId` | Not used (TUI sessions are linear) |
|
||||
| `replyToId` | Not used |
|
||||
| `attachments` | File paths dragged/pasted into the TUI; resolved to `file://` URLs |
|
||||
| `timestamp` | `new Date()` at the moment of send |
|
||||
| `metadata` | `{ conversationId, sessionId, ttyWidth, colorSupport }` |
|
||||
|
||||
**Outbound:** The adapter writes rendered content to stdout. Markdown is rendered via a terminal markdown renderer (e.g. `marked-terminal`). Code blocks are syntax-highlighted when `metadata.colorSupport = true`.
|
||||
|
||||
---
|
||||
|
||||
### WebUI
|
||||
|
||||
The WebUI adapter connects the Next.js frontend (`apps/web`) to the channel protocol over the existing Socket.IO gateway (`apps/gateway`).
|
||||
|
||||
| ChannelMessageDto field | WebUI equivalent |
|
||||
| ----------------------- | ------------------------------------------------------------ |
|
||||
| `id` | Generated UUID; echoed back in the WebSocket event |
|
||||
| `channelId` | `"webui:<conversationId>"` |
|
||||
| `senderId` | Authenticated Mosaic `userId` |
|
||||
| `senderKind` | `"user"` for browser input; `"agent"` for agent responses |
|
||||
| `content` | Message text from the input field |
|
||||
| `contentKind` | `"text"` or `"markdown"` |
|
||||
| `threadId` | Not used (conversation model handles threading) |
|
||||
| `replyToId` | Message ID the user replied to (UI reply affordance) |
|
||||
| `attachments` | Files uploaded via the file picker; stored to object storage |
|
||||
| `timestamp` | `new Date()` at send, or server timestamp from event |
|
||||
| `metadata` | `{ conversationId, sessionId, clientTimezone, userAgent }` |
|
||||
|
||||
**Outbound:** Adapter emits a `chat:message` Socket.IO event. The WebUI React component receives it and appends to the conversation list. Markdown content is rendered client-side via the existing markdown renderer component.
|
||||
|
||||
---
|
||||
|
||||
## Identity Mapping
|
||||
|
||||
Gateway identity-linking policy resolves a channel-native user identifier to a Mosaic `userId` and produces `ChannelAuthorizedPrincipalDto`. Adapters provide native identity evidence but cannot self-authorize Mosaic scope. Discord currently uses configuration-owned paired users; database-backed linking remains the canonical direction for dynamic Matrix/Slack identity.
|
||||
|
||||
The implementation must query a `channel_identities` table (or equivalent) keyed on `(channel_name, channel_user_id)`. When no mapping exists the method returns `null` and the message is treated as anonymous (no Mosaic session context).
|
||||
|
||||
```
|
||||
channel_identities
|
||||
channel_name TEXT -- e.g. "matrix", "discord"
|
||||
channel_user_id TEXT -- channel-native user identifier
|
||||
mosaic_user_id TEXT -- FK to users.id
|
||||
linked_at TIMESTAMP
|
||||
PRIMARY KEY (channel_name, channel_user_id)
|
||||
```
|
||||
|
||||
Identity linking flows (OAuth dance, deep-link verification token, etc.) are out of scope for this document and will be specified in a separate identity-linking protocol document.
|
||||
|
||||
---
|
||||
|
||||
## Error Handling Conventions
|
||||
|
||||
- `start()` must establish the native channel transport or throw a structured connection error. An adapter hosted inside the gateway must not wait for a loopback connection to that same not-yet-listening process; it starts the native transport, lets Socket.IO reconnect, and reports `degraded` until both links are ready.
|
||||
- `ChannelEgressPort.send()` implementations must throw a typed terminal error for revoked auth, an invalid route, or a missing channel. Only transient rate/network/server failures are retried with bounded exponential backoff; Discord retries reuse a stable enforced nonce to prevent duplicate chunks, while permanent 4xx failures are not retried.
|
||||
- `health()` must never throw — it returns `{ status: 'disconnected' }` on error.
|
||||
- Adapters must emit structured logs with `{ channel: adapter.name, event, ... }` metadata for observability.
|
||||
|
||||
---
|
||||
|
||||
## Versioning
|
||||
|
||||
The `ChannelMessageDto` protocol follows semantic versioning. Non-breaking field additions (new optional fields) are minor version bumps. Breaking changes (type changes, required field additions) require a major version bump and a migration guide.
|
||||
|
||||
Current version: **1.0.0**
|
||||
|
||||
---
|
||||
|
||||
## M7-003: Matrix Integration Design
|
||||
|
||||
### Homeserver Choice
|
||||
|
||||
Mosaic uses **Conduit** as the Matrix homeserver. Conduit is written in Rust, ships as a single binary, and has minimal operational overhead compared to Synapse or Dendrite. It supports the full Matrix Client-Server and Application Service APIs required by Mosaic.
|
||||
|
||||
Recommended deployment: Conduit runs as a Docker container alongside the Mosaic stack. A single Conduit instance is sufficient for most self-hosted deployments. Conduit's embedded RocksDB storage means no separate database is required for the homeserver itself.
|
||||
|
||||
### Appservice Registration
|
||||
|
||||
Mosaic registers with the Conduit homeserver as a Matrix **Application Service (appservice)**. This gives Mosaic the ability to:
|
||||
|
||||
- Create and control ghost users (virtual Matrix users representing Mosaic agents and provisioned accounts).
|
||||
- Receive all events sent to rooms within the appservice's namespace without polling.
|
||||
- Send events on behalf of ghost users without separate authentication.
|
||||
|
||||
Registration is done via a YAML registration file (`mosaic-appservice.yaml`) placed in Conduit's configuration directory:
|
||||
|
||||
```yaml
|
||||
id: mosaic
|
||||
url: http://gateway:3000/_matrix/appservice
|
||||
as_token: <random-secret>
|
||||
hs_token: <random-secret>
|
||||
sender_localpart: mosaic-bot
|
||||
namespaces:
|
||||
users:
|
||||
- exclusive: true
|
||||
regex: '@mosaic_.*:homeserver'
|
||||
rooms:
|
||||
- exclusive: false
|
||||
regex: '.*'
|
||||
aliases:
|
||||
- exclusive: true
|
||||
regex: '#mosaic-.*:homeserver'
|
||||
```
|
||||
|
||||
The gateway exposes `/_matrix/appservice` endpoints to receive push events from Conduit. The `as_token` and `hs_token` are stored in Vault and injected at startup.
|
||||
|
||||
### Room ↔ Conversation Mapping
|
||||
|
||||
Each Mosaic conversation maps to a single Matrix room. The mapping is stored in the database:
|
||||
|
||||
```
|
||||
conversation_matrix_rooms
|
||||
conversation_id TEXT -- FK to conversations.id
|
||||
room_id TEXT -- Matrix room ID (!roomid:homeserver)
|
||||
created_at TIMESTAMP
|
||||
PRIMARY KEY (conversation_id)
|
||||
```
|
||||
|
||||
Room creation is handled by the appservice on the first Matrix access to a conversation. Room names follow the pattern `Mosaic: <conversation title>`. Room topics contain the conversation ID for correlation.
|
||||
|
||||
When a conversation is deleted or archived in Mosaic, the corresponding Matrix room is tombstoned (m.room.tombstone event) and the room is left in a read-only state.
|
||||
|
||||
### Space ↔ Team Mapping
|
||||
|
||||
Each Mosaic team maps to a Matrix **Space**. Spaces are Matrix rooms with a special `m.space` type that can contain child rooms.
|
||||
|
||||
```
|
||||
team_matrix_spaces
|
||||
team_id TEXT -- FK to teams.id
|
||||
space_id TEXT -- Matrix room ID of the Space
|
||||
created_at TIMESTAMP
|
||||
PRIMARY KEY (team_id)
|
||||
```
|
||||
|
||||
When a conversation room is shared with a team, the appservice adds it to the team's Space via `m.space.child` state events. Removing the share removes the child relationship.
|
||||
|
||||
### Agent Ghost Users
|
||||
|
||||
Each Mosaic agent is represented in Matrix as an **appservice ghost user**:
|
||||
|
||||
- Matrix user ID format: `@mosaic_agent_<agentId>:homeserver`
|
||||
- Display name: the agent's human-readable name (e.g. "Mosaic Assistant")
|
||||
- Avatar: optional, configurable per agent
|
||||
|
||||
Ghost users are registered lazily — the appservice creates the ghost on first use. Ghost users are controlled exclusively by the appservice; they cannot log in via Matrix client credentials.
|
||||
|
||||
When an agent sends a message via the gateway, the Matrix adapter sends the event using `user_id` impersonation on the appservice's client endpoint, causing the message to appear as if sent by the ghost user.
|
||||
|
||||
### Power Levels
|
||||
|
||||
Power levels in each Mosaic-managed room are set as follows:
|
||||
|
||||
| Entity | Power Level | Rationale |
|
||||
| ------------------------------------- | -------------- | -------------------------------------- |
|
||||
| Mosaic appservice bot (`@mosaic-bot`) | 100 (Admin) | Room management and moderation |
|
||||
| Human Mosaic users | 50 (Moderator) | Can kick, redact, and invite |
|
||||
| Agent ghost users | 0 (Default) | Message-only; cannot modify room state |
|
||||
|
||||
This arrangement ensures human users retain full control. An agent cannot modify room settings, kick members, or take administrative actions. Humans with moderator power can redact agent messages and intervene in ongoing conversations.
|
||||
|
||||
```
|
||||
mermaid
|
||||
graph TD
|
||||
A[Mosaic Admin] -->|invites| B[Human User]
|
||||
B -->|joins| C[Matrix Room / Conversation]
|
||||
D[Agent Ghost User] -->|sends messages to| C
|
||||
B -->|can redact/kick| D
|
||||
E[Mosaic Bot] -->|manages room state| C
|
||||
style A fill:#4a9eff
|
||||
style B fill:#4a9eff
|
||||
style D fill:#aaaaaa
|
||||
style E fill:#ff9944
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## M7-004: Conversation Multiplexing
|
||||
|
||||
### Architecture Overview
|
||||
|
||||
A single Mosaic conversation can be accessed simultaneously from multiple surfaces: TUI, WebUI, and Matrix. The gateway is the **single source of truth** for all conversation state. Each surface is a thin client that renders gateway-owned data.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ Gateway (NestJS) │
|
||||
│ │
|
||||
│ ConversationService ←→ MessageBus │
|
||||
│ │ │ │
|
||||
│ [DB: PostgreSQL] [Fanout: Valkey Pub/Sub] │
|
||||
│ │ │
|
||||
│ ┌─────────────────────┼──────────────┐ │
|
||||
│ │ │ │ │
|
||||
│ Socket.IO Socket.IO Matrix │ │
|
||||
│ (TUI adapter) (WebUI adapter) (appservice)│ │
|
||||
└──────────┼─────────────────────┼──────────────┘ │
|
||||
│ │ │
|
||||
CLI/TUI Browser Matrix
|
||||
Client
|
||||
```
|
||||
|
||||
### Real-Time Sync Flow
|
||||
|
||||
1. A message arrives on any surface (TUI keystroke, browser send, Matrix event).
|
||||
2. The surface's adapter normalizes the message to `ChannelMessageDto` and delivers it to `ConversationService`.
|
||||
3. `ConversationService` persists the message to PostgreSQL, assigns a canonical `id`, and publishes a `message:new` event to the Valkey pub/sub channel keyed by `conversationId`.
|
||||
4. All active surfaces subscribed to that `conversationId` receive the fanout event and push it to their respective clients:
|
||||
- TUI adapter: writes rendered output to the connected terminal session.
|
||||
- WebUI adapter: emits a `chat:message` Socket.IO event to all browser sessions joined to that conversation.
|
||||
- Matrix adapter: sends an `m.room.message` event to the conversation's Matrix room.
|
||||
|
||||
This ensures that a message typed in the TUI appears in the browser and in Matrix within the same round-trip latency as the Valkey fanout (typically <10 ms on co-located infrastructure).
|
||||
|
||||
### Surface-to-Transport Mapping
|
||||
|
||||
| Surface | Transport to Gateway | Fanout Transport from Gateway |
|
||||
| ------- | ------------------------------------------ | ----------------------------- |
|
||||
| TUI | HTTPS REST + SSE or WebSocket | Socket.IO over stdio proxy |
|
||||
| WebUI | Socket.IO (browser) | Socket.IO emit |
|
||||
| Matrix | Matrix Client-Server API (appservice push) | Matrix `m.room.message` send |
|
||||
|
||||
### Conflict Resolution
|
||||
|
||||
- **Messages**: Append-only. Messages are never edited in-place in Mosaic's canonical store. Matrix edit events (`m.replace`) are treated as new messages with `replyToId` pointing to the original, preserving the full audit trail.
|
||||
- **Metadata (title, tags, archived state)**: Last-write-wins. The timestamp of the most recent write wins. Concurrent metadata updates from different surfaces are serialized through `ConversationService`; the final database write reflects the last persisted value.
|
||||
- **Conversation membership**: Set-merge semantics. Adding a user from any surface is additive. Removal requires an explicit delete action and is not overwritten by concurrent adds.
|
||||
|
||||
### Session Isolation
|
||||
|
||||
Multiple TUI sessions or browser tabs connected to the same conversation receive all fanout messages independently. Each session maintains its own scroll position and local ephemeral state (typing indicator, draft text). Gateway does not synchronize ephemeral state across sessions.
|
||||
|
||||
---
|
||||
|
||||
## M7-005: Remote Auth Bridging
|
||||
|
||||
### Overview
|
||||
|
||||
Matrix users authenticate to Mosaic by linking their Matrix identity to an existing Mosaic account. There are two flows: token linking (primary) and OAuth bridge (alternative). Once linked, the Matrix session is persistent — there is no periodic login/logout cycle.
|
||||
|
||||
### Token Linking Flow
|
||||
|
||||
1. A Mosaic admin or the user themselves generates a short-lived link token via the Mosaic web UI or API (`POST /auth/channel-link-token`). The token is a cryptographically random 32-byte hex string with a 15-minute TTL stored in Valkey.
|
||||
2. The user opens a Matrix client and sends a DM to `@mosaic-bot:homeserver`.
|
||||
3. The user sends the command: `!link <token>`
|
||||
4. The appservice receives the `m.room.message` event in the DM room, extracts the token, and calls `AuthService.linkChannelIdentity({ channel: 'matrix', channelUserId: matrixUserId, token })`.
|
||||
5. `AuthService` validates the token, retrieves the associated `mosaicUserId`, and writes a row to `channel_identities`.
|
||||
6. The appservice sends a confirmation reply in the DM room and invites the now-linked user to their personal Matrix Space.
|
||||
|
||||
```
|
||||
User (Matrix) @mosaic-bot Mosaic Gateway
|
||||
│ │ │
|
||||
│ DM: !link <token> │ │
|
||||
│────────────────────▶│ │
|
||||
│ │ POST /auth/link │
|
||||
│ │─────────────────────▶│
|
||||
│ │ 200 OK │
|
||||
│ │◀─────────────────────│
|
||||
│ ✓ Linked! Joining │ │
|
||||
│ your Space now │ │
|
||||
│◀────────────────────│ │
|
||||
```
|
||||
|
||||
### OAuth Bridge Flow
|
||||
|
||||
An alternative flow for users who prefer browser-based authentication:
|
||||
|
||||
1. The Mosaic bot sends the user a Matrix message containing an OAuth URL: `https://mosaic.example.com/auth/matrix-link?state=<nonce>&matrix_user=<encoded_mxid>`
|
||||
2. The user opens the URL in a browser. If not already logged in to Mosaic, they are redirected through the standard BetterAuth login flow.
|
||||
3. On successful authentication, Mosaic records the `channel_identities` row linking `matrix_user` to the authenticated `mosaicUserId`.
|
||||
4. The gateway sends a Matrix event to the pending DM room confirming the link.
|
||||
|
||||
### Invite-Based Provisioning
|
||||
|
||||
When a Mosaic admin adds a new user account, the provisioning flow optionally associates a Matrix user ID with the new account at creation time:
|
||||
|
||||
1. Admin provides `matrixUserId` when creating the account (`POST /admin/users`).
|
||||
2. `UserService` writes the `channel_identities` row immediately.
|
||||
3. The Matrix adapter's provisioning hook fires, and the appservice:
|
||||
- Creates the user's personal Matrix Space (if not already existing).
|
||||
- Sends an invite to the Matrix user for their personal Space.
|
||||
- Sends a welcome DM from `@mosaic-bot` with onboarding instructions.
|
||||
|
||||
The invited user does not need to complete any linking step — the association is pre-established by the admin.
|
||||
|
||||
### Session Lifecycle
|
||||
|
||||
Matrix sessions for linked users are persistent and long-lived. Unlike TUI sessions (which terminate when the terminal process exits), a Matrix user's access to their rooms remains intact as long as:
|
||||
|
||||
- Their Mosaic account is active (not suspended or deleted).
|
||||
- Their `channel_identities` row exists (link not revoked).
|
||||
- They remain members of the relevant Matrix rooms.
|
||||
|
||||
Revoking a Matrix link (`DELETE /auth/channel-link/matrix/<matrixUserId>`) removes the `channel_identities` row and causes gateway principal resolution to deny the identity. The appservice optionally kicks the Matrix user from all Mosaic-managed rooms as part of the revocation flow (configurable, default: off).
|
||||
|
||||
---
|
||||
|
||||
## M7-006: Agent-to-Agent Communication via Matrix
|
||||
|
||||
### Dedicated Agent Rooms
|
||||
|
||||
When two Mosaic agents need to coordinate, a dedicated Matrix room is created for their dialogue. This provides a persistent, auditable channel for structured inter-agent communication that humans can observe.
|
||||
|
||||
Room naming convention:
|
||||
|
||||
```
|
||||
#mosaic-agents-<agentA>-<agentB>:homeserver
|
||||
```
|
||||
|
||||
Where `agentA` and `agentB` are the Mosaic agent IDs sorted lexicographically (to ensure the same room is used regardless of which agent initiates). The room alias is registered by the appservice.
|
||||
|
||||
```
|
||||
agent_rooms
|
||||
room_id TEXT -- Matrix room ID
|
||||
agent_a_id TEXT -- FK to agents.id (lexicographically first)
|
||||
agent_b_id TEXT -- FK to agents.id (lexicographically second)
|
||||
created_at TIMESTAMP
|
||||
PRIMARY KEY (agent_a_id, agent_b_id)
|
||||
```
|
||||
|
||||
### Room Membership and Power Levels
|
||||
|
||||
| Entity | Power Level |
|
||||
| ---------------------------------- | ------------------------------------ |
|
||||
| Mosaic appservice bot | 100 (Admin) |
|
||||
| Human observers (invited) | 50 (Moderator, read-only by default) |
|
||||
| Agent ghost users (agentA, agentB) | 0 (Default — message send only) |
|
||||
|
||||
Humans are invited to agent rooms with a read-only intent. By convention, human messages in agent rooms are prefixed with `[HUMAN]` and treated as interrupts by the gateway. Agents are instructed (via system prompt) to pause and acknowledge human messages before resuming their dialogue.
|
||||
|
||||
### Message Format
|
||||
|
||||
Agents communicate using **structured JSON** embedded in Matrix event content. The Matrix event type is `m.room.message` with `msgtype: "m.text"` for compatibility. The structured payload is carried in a custom `mosaic.agent_message` field:
|
||||
|
||||
```json
|
||||
{
|
||||
"msgtype": "m.text",
|
||||
"body": "[Agent message — see mosaic.agent_message for structured content]",
|
||||
"mosaic.agent_message": {
|
||||
"schema_version": "1.0",
|
||||
"sender_agent_id": "agent_abc123",
|
||||
"conversation_id": "conv_xyz789",
|
||||
"message_type": "request",
|
||||
"payload": {
|
||||
"action": "summarize",
|
||||
"parameters": { "max_tokens": 500 },
|
||||
"reply_to_event_id": "$previousEventId"
|
||||
},
|
||||
"timestamp_ms": 1711234567890
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `body` field contains a human-readable fallback so the conversation is legible in any Matrix client. The structured payload is parsed exclusively by the gateway's Matrix adapter.
|
||||
|
||||
### Coordination Patterns
|
||||
|
||||
**Request/Response**: Agent A sends a `message_type: "request"` event. Agent B sends a `message_type: "response"` with `reply_to_event_id` referencing Agent A's event. The gateway correlates request/response pairs using the event IDs.
|
||||
|
||||
**Broadcast**: An agent sends a `message_type: "broadcast"` to a multi-agent room (more than two members). All agents in the room receive the event. No response is expected.
|
||||
|
||||
**Delegation**: Agent A sends a `message_type: "delegate"` with a `payload.task` object describing work to be handed off to Agent B. Agent B acknowledges with `message_type: "delegate_ack"` and later sends `message_type: "delegate_complete"` when done.
|
||||
|
||||
```
|
||||
AgentA Gateway AgentB
|
||||
│ delegate(task) │ │
|
||||
│────────────────────▶│ │
|
||||
│ │ Matrix event push │
|
||||
│ │────────────────────▶│
|
||||
│ │ delegate_ack │
|
||||
│ │◀────────────────────│
|
||||
│ │ [AgentB executes] │
|
||||
│ │ delegate_complete │
|
||||
│ │◀────────────────────│
|
||||
│ task result │ │
|
||||
│◀────────────────────│ │
|
||||
```
|
||||
|
||||
### Gateway Mediation
|
||||
|
||||
Agents do not call the Matrix Client-Server API directly. All inter-agent Matrix events are sent and received by the gateway's appservice. This means:
|
||||
|
||||
- The gateway can intercept, log, and rate-limit agent-to-agent messages.
|
||||
- Agents that are offline (no active process) still have their messages delivered; the gateway queues them and delivers on the agent's next activation.
|
||||
- The gateway can inject system messages (e.g. human interrupts, safety stops) into agent rooms without agent cooperation.
|
||||
|
||||
---
|
||||
|
||||
## M7-007: Multi-User Isolation in Matrix
|
||||
|
||||
### Space-per-Team Architecture
|
||||
|
||||
Isolation in Matrix is enforced through the Space hierarchy. Each organizational boundary in Mosaic maps to a distinct Matrix Space:
|
||||
|
||||
| Mosaic entity | Matrix Space | Visibility |
|
||||
| ----------------------------- | -------------- | ----------------- |
|
||||
| Personal workspace (per user) | Personal Space | User only |
|
||||
| Team | Team Space | Team members only |
|
||||
| Public project | (no Space) | Configurable |
|
||||
|
||||
Rooms (conversations) are placed into Spaces based on their sharing configuration. A room can appear in at most one team Space at a time. Moving a room from one team Space to another removes the `m.space.child` link from the old Space and adds it to the new one.
|
||||
|
||||
### Room Visibility Rules
|
||||
|
||||
Matrix room visibility within Conduit is controlled by:
|
||||
|
||||
1. **Join rules**: All Mosaic-managed rooms use `join_rule: invite`. Users cannot discover or join rooms without an explicit invite from the appservice.
|
||||
2. **Space membership**: Rooms appear in a Space's directory only to users who are members of that Space.
|
||||
3. **Room directory**: The server room directory is disabled for Mosaic-managed rooms (`m.room.history_visibility: shared` for team rooms, `m.room.history_visibility: invited` for personal rooms).
|
||||
|
||||
### Personal Space Defaults
|
||||
|
||||
When a user account is created (or linked to Matrix), the appservice provisions a personal Space:
|
||||
|
||||
- Space name: `<username>'s Space`
|
||||
- All conversations the user creates personally are added as children of their personal Space.
|
||||
- No other users are members of this Space by default.
|
||||
- Conversation rooms within the personal Space are only visible and accessible to the owner.
|
||||
|
||||
### Team Shared Rooms
|
||||
|
||||
When a project or conversation is shared with a team:
|
||||
|
||||
1. The appservice adds the room as a child of the team's Space (`m.space.child` state event in the Space room, `m.space.parent` state event in the conversation room).
|
||||
2. All current team members are invited to the conversation room.
|
||||
3. Newly added team members are automatically invited to all shared rooms in the team's Space by the appservice's team membership hook.
|
||||
4. If sharing is revoked, the appservice removes the `m.space.child` link and kicks all team members who joined via the team share (users who were directly invited are unaffected).
|
||||
|
||||
### Encryption
|
||||
|
||||
Encryption is optional and configured per room at creation time. Recommended defaults:
|
||||
|
||||
| Space type | Encryption default | Rationale |
|
||||
| -------------- | ------------------ | -------------------------------------- |
|
||||
| Personal Space | Enabled | Privacy-first for individual users |
|
||||
| Team Space | Disabled | Operational visibility; admin auditing |
|
||||
| Agent rooms | Disabled | Gateway must read structured payloads |
|
||||
|
||||
When encryption is enabled, the appservice's ghost users must participate in key exchange (using Matrix's Olm/Megolm protocol). The gateway holds the device keys for all ghost users it controls. This constraint means encrypted rooms require the gateway to be the E2E session holder — messages are end-to-end encrypted between human clients and gateway-held ghost device keys, not between human clients themselves.
|
||||
|
||||
### Admin Visibility
|
||||
|
||||
A Conduit server administrator can see:
|
||||
|
||||
- Room metadata: names, aliases, topic, membership list.
|
||||
- Unencrypted event content in unencrypted rooms.
|
||||
|
||||
A Conduit server administrator **cannot** see:
|
||||
|
||||
- Content of encrypted rooms (without holding a device key for a room member).
|
||||
|
||||
Mosaic does not grant gateway admin credentials to application-level admin users. The Conduit admin interface is restricted to infrastructure operators. Application-level admins manage users and rooms through the Mosaic API, which interacts with the appservice layer only.
|
||||
|
||||
### Data Retention
|
||||
|
||||
Matrix events in Mosaic-managed rooms follow Mosaic's configurable retention policy:
|
||||
|
||||
```
|
||||
room_retention_policies
|
||||
room_id TEXT -- Matrix room ID (or wildcard pattern)
|
||||
retention_days INT -- NULL = keep forever
|
||||
applies_to TEXT -- "personal" | "team" | "agent" | "all"
|
||||
created_at TIMESTAMP
|
||||
```
|
||||
|
||||
The retention policy is enforced by a background job in the gateway that calls Conduit's admin API to purge events older than the configured threshold. Purged events are removed from the Conduit store but Mosaic's PostgreSQL message store retains the canonical `ChannelMessageDto` record unless the Mosaic retention policy also covers it.
|
||||
|
||||
Default retention values:
|
||||
|
||||
| Room type | Default retention |
|
||||
| --------------------------- | ------------------- |
|
||||
| Personal conversation rooms | 365 days |
|
||||
| Team conversation rooms | 730 days |
|
||||
| Agent-to-agent rooms | 90 days |
|
||||
| System/audit rooms | 1825 days (5 years) |
|
||||
|
||||
Retention settings are configurable by Mosaic admins via the admin API and apply to both the Matrix event store and the Mosaic message store in lockstep.
|
||||
@@ -0,0 +1,59 @@
|
||||
# Compaction observer revocation and runtime generations
|
||||
|
||||
WI-3 connects Claude and Pi compaction/session lifecycle events to the existing authenticated lease-broker state machine. It does not add a second lease store or let runtime hooks assert identity. Each observer inherits the broker-minted session, resolves the current private runtime generation, and sends the existing `revoke_lease` action over the authenticated Unix socket.
|
||||
|
||||
## Observer matrix
|
||||
|
||||
| Runtime | Lifecycle signal | Action |
|
||||
| ---------------- | ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Claude / Claudex | `PreCompact` | Revoke the current lease before compaction. A non-zero hook result blocks the lifecycle transition. |
|
||||
| Claude / Claudex | `SessionStart` with matcher `compact` | Revoke again after compacted context starts. |
|
||||
| Claude / Claudex | `SessionStart` with matcher `resume\|clear` | Atomically advance the private generation, then revoke the replacement incarnation. |
|
||||
| Pi | `session_before_compact` | Revoke before compaction; return `{ cancel: true }` if revocation cannot be confirmed. |
|
||||
| Pi | `session_compact` then the first `context` | Arm and run an independent post-compaction revoke. A failed post observer blocks later tools locally until a retry succeeds. |
|
||||
| Pi | `session_start` with reason `reload`, `new`, `resume`, or `fork` | Atomically advance the private generation, then revoke the replacement incarnation before reuse. |
|
||||
|
||||
The first observer that reaches the broker deletes pending promotion tokens and makes the lease `UNVERIFIED`. The second compaction observer is deliberate redundancy, not a prerequisite for the first. Claudex receives the same mandatory hooks in its isolated `CLAUDE_CONFIG_DIR`; hook merging preserves unrelated isolated settings and rejects malformed or symlinked settings fail-closed.
|
||||
|
||||
## Private generation authority
|
||||
|
||||
`launch-runtime.py` still registers before `exec`, preserving the kernel-authenticated PID/starttime anchor. It now also creates `generation-<broker-session>.state` beside the broker socket. The file is owner-only mode `0600` under the broker's mode-`0700` directory. Hook descendants read that file instead of relying only on an immutable inherited environment value.
|
||||
|
||||
Generation changes use an exclusive file lock, validate owner/type/mode/size, increment monotonically, truncate and write the complete new value, and `fsync` before contacting the broker. Therefore reload, new-session, resume, and fork events may retain the same PID/starttime while still becoming a new broker incarnation. The higher generation causes the broker to atomically discard prior tokens and lease authority; the replacement generation inherits no VERIFIED lease.
|
||||
|
||||
If an observer fires while broker transport is unavailable, `revoke-lease.py` advances the private generation as a local fence before returning non-zero. Every later all-tools gate reads that higher value. When the broker is reachable again, authentication of that value performs the same old-generation revocation before authorization. Pi also keeps a process-local post-compaction/rollover failure latch that blocks tool calls. An unsafe or unreadable generation file itself makes both lifecycle revocation and tool authorization fail closed.
|
||||
|
||||
## Threat contract and stopping boundary
|
||||
|
||||
### BOUNDED RESIDUAL STALE WINDOW
|
||||
|
||||
If **both** pre- and post-compaction observers are missed entirely, no revocation signal exists. During the remaining unexpired lease, **within-TTL consequential actions are allowed**. Their count and timing are **bounded by lease expiry, not by the mutator gate**. WI-3 makes no claim that it bounds mutator actions inside this stale interval. The broker's monotonic lease TTL is capped at 300 seconds; after expiry, the next consequential tool is denied with `LEASE_EXPIRED`.
|
||||
|
||||
This is the named D2-v5 T-A residual. It is distinct from an observer that fires but cannot contact the broker: the latter creates a local generation fence and fails closed. It is also distinct from T-C total rot, where the lifecycle observers and the all-tools gate are both absent or replaced. Server-side branch protection, required CI, and independent review remain the irreducible backstop for T-C.
|
||||
|
||||
| Condition | Result |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
|
||||
| Either compaction observer succeeds | Existing lease and pending promotion tokens are revoked immediately. |
|
||||
| Observer runs but broker confirmation fails | Lifecycle transition is denied where supported; local generation fence and runtime latch prevent inherited authority. |
|
||||
| Both observers are missed, lease unexpired | **ALLOWED** inside the bounded residual stale window. No within-window mutator bound is claimed. |
|
||||
| Both observers are missed, lease expired | **DENIED** by monotonic TTL expiry. |
|
||||
| Generation advances on reload/new/resume/fork | Prior incarnation revoked; replacement starts `UNVERIFIED`. |
|
||||
| Lifecycle observers and all-tools gate both fail or are removed | T-C total-hook-miss residual; protected-branch controls remain required. |
|
||||
|
||||
## T-C server-side branch-protection posture
|
||||
|
||||
The required posture is that `main` is push-blocked and PR-only-merge is **MANDATORY**, regardless
|
||||
of client-gate state. The client-side gate narrows the exposure window only; it is not the T-C
|
||||
guarantee. The server-side protected-branch configuration is the irreducible guarantee for protected
|
||||
repository actions. Status-check enforcement and approval enforcement are **RECOMMENDED**.
|
||||
|
||||
## Current-vs-required gap (recorded, not enacted)
|
||||
|
||||
The current empirical configuration is recorded here without re-probing or mutating live branch
|
||||
protection. `enable_push=False` (push-block present), so the mandatory push-block/PR-only-merge core
|
||||
holds. `require_approvals=0` (approvals not enforced), `enable_status_check=False` (status checks not
|
||||
enforced), and `block_on_official_review=False` (official review not enforced). Those recommended
|
||||
merge-quality controls are the current gap; changing them is a separate, owner-gated operations
|
||||
decision and is not enacted by this documentation change.
|
||||
|
||||
The permanent T12b/T30 acceptance case prints both required outcomes: dual-hook miss within TTL is **ALLOWED**, and the same lease after TTL is **DENIED**. Separate real-socket tests prove each Claude observer and same-PID generation rollover; Pi lifecycle tests exercise pre/post observers, all four replacement reasons, and local failure closure.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Authenticated external lease broker protocol
|
||||
|
||||
The compaction-refresh lease broker is a Linux-only, newline-framed JSON protocol over a Unix stream socket. It is runtime-neutral; M1 consumers are limited to Claude and Pi. This is an internal process boundary, not an HTTP API, so it is intentionally absent from OpenAPI.
|
||||
|
||||
The broker, never the caller, obtains `(pid, uid, gid)` from kernel `SO_PEERCRED`. It correlates the PID with `/proc/<pid>/stat` field 22 (`starttime`) and mints `session_id` on `register_anchor`. Presence of `session_id` in that request is refused even when its value is `null` or empty. Later requests must originate from the anchor or a descendant. The broker walks parent PIDs to the `(pid,starttime)` anchor and then rereads every walked PID's starttime before accepting the chain.
|
||||
|
||||
## Request and response boundary
|
||||
|
||||
Each connection carries exactly one UTF-8 JSON object followed by one newline, capped at 64 KiB. The protocol deliberately uses EOF to prove that there is exactly one frame: immediately after writing the newline, the client **MUST half-close its write side** with `shutdown(SHUT_WR)` (or Node `socket.end()`) before awaiting the response. A client that writes a newline but leaves its write side open receives no successful response; the broker's one-second connection deadline fails closed. Malformed, unterminated, multiple (including a delayed second frame), or oversized frames fail closed. Responses are one JSON object and one newline. Success has `{"ok":true,...}`; refusal has `{"ok":false,"code":"TYPED_CODE"}`. Requests are:
|
||||
|
||||
- `register_anchor`: `action`, non-negative `runtime_generation`; no `session_id` field.
|
||||
- `authenticate`: `action`, broker-minted `session_id`, non-negative `runtime_generation`.
|
||||
- `mint_token`: authenticated identity plus `binding` containing exactly `compaction_epoch`, `request_epoch`, `h_source`, `h_payload`, and `schema_version`.
|
||||
- `consume_token`: authenticated identity plus `token`.
|
||||
- `begin_verification`: authenticated identity, runtime (`claude` or `pi`), cycle `binding`, and a TTL no greater than 300 seconds. The broker revokes existing authority first, enters `PENDING_VERIFICATION`, and returns a single-use promotion token.
|
||||
- `begin_recovery`: the constrained recovery entrypoint. It rejects caller-provided receipt/challenge fields and delegates to the same `begin_verification` transition, but reports `PENDING_DELIVERY` and marks the volatile cycle as recovery-owned.
|
||||
- `complete_recovery`: authenticated identity only. It rejects caller-provided receipt/challenge fields, obtains the current recovery challenge only from broker state, and delegates to the same trusted-observer → evidence → consume → promote sequence. An observation failure revokes recovery authority; retry starts a fresh challenge.
|
||||
|
||||
The daemon owns a second protected production observer socket (mode `0600`) unless a private `--test-observer-file` fixture is selected. That transport accepts only the exact `record_runtime_observation` schema after kernel `SO_PEERCRED` plus the existing anchor/ancestry authentication; it validates the pending runtime/generation before storing one finalized assistant entry for the in-process `RuntimeReceiptObserver`. It is **not** a broker request action. Claude sends its latest assistant entry from the Stop-hook transport; Pi sends only finalized `message_end` assistant content. The public broker socket continues to reject request-supplied `latest_assistant_message` in begin, observe, and complete paths.
|
||||
|
||||
- `promote_lease`: authenticated identity plus the exact pending promotion token. The broker commits token consumption before making `VERIFIED` visible.
|
||||
- `revoke_lease`: authenticated observer signal; deletes pending tokens and makes the session `UNVERIFIED` immediately. WI-3 Claude/Pi hooks send this existing action; `runtime` and bounded `reason` fields are diagnostic input only and never identity authority.
|
||||
- `authorize_tool`: authenticated identity, runtime, and exact runtime-reported tool name. The broker returns an explicit allow/deny decision from the whole-class policy and current lease.
|
||||
|
||||
A higher generation for the same anchor atomically replaces the stored incarnation and deletes all prior tokens and lease authority for that session. A lower generation is stale. Runtime descendants resolve the current generation from an owner-only, locked generation file created by the register-before-exec launcher; reload/new/resume/fork observers advance and `fsync` it before broker revocation. This supports generation replacement even when PID/starttime do not change. Tokens are 256-bit values from the operating-system cryptographic RNG and are single use. At most 256 pending tokens may be persisted; another mint fails with `TOKEN_CAPACITY` before mutation. Successful consumption deletes the token, while a replay still fails with `TOKEN_REPLAY`. Live v1 token records retain the existing `consumed: false` schema.
|
||||
|
||||
VERIFIED leases are volatile and monotonic-time bounded: broker restart, generation change, explicit observer revocation, or expiry returns the session to `UNVERIFIED`. `begin_verification` always revokes before minting a new prerequisite. `begin_recovery` reuses that exact transition and mints a new challenge, so a normal-path receipt/challenge cannot be replayed through recovery. `promote_lease` is valid only from the matching pending cycle; persistence failure rolls token and lease state back, while post-rename durability uncertainty terminates the broker. The WI-1 token is the atomic promotion prerequisite substrate.
|
||||
|
||||
## Receipt boundary and T-C residual (R1)
|
||||
|
||||
Receipt evidence is a T-A delivery/liveness prerequisite only; it cannot replace the mechanical
|
||||
mutator gate as safety authority. The receipt detects an **ABSENT** or **PREFIX-TRUNCATED** terminal
|
||||
token. A **MIDDLE-DROP** that preserves the tail is a T-C contract violation that is **NOT receipt-detectable**. It is covered by server-side protected-branch controls, **NOT** by the receipt; no category-wide receipt-detection claim is made for that tail-preserving transformation.
|
||||
|
||||
State replacement serializes and enforces the 4 MiB maximum before opening a temporary file, then uses a mode-`0600` temporary file, `fsync`, atomic rename, and parent-directory `fsync`. Every broker mutation snapshots the prior v1 state. A commit failure before rename restores that snapshot and leaves durable state unchanged. A failure after rename makes durability uncertain, so the store is poisoned without rolling memory back and the daemon terminates rather than serving with divergent state. Existing state is opened without following symlinks, must be a bounded regular file at mode `0600`, and is fully schema- and invariant-validated before use. Persisted tokens must be unconsumed, match their session's current generation, and remain within the 256-token cap. Session identity is uniquely keyed by `(anchor_pid,anchor_starttime)`; duplicate logical sessions for one anchor refuse startup. State integrity or mode failures refuse startup. The daemon does not log session IDs or tokens.
|
||||
@@ -0,0 +1,26 @@
|
||||
# WI-1 lease broker security notes
|
||||
|
||||
- Trusted identity comes only from Linux `SO_PEERCRED` plus `/proc` starttime, never request identity fields.
|
||||
- Descendant authorization is anchored to `(pid,starttime)` and uses a complete second starttime pass to fail closed on disappearance or PID-reuse races.
|
||||
- Runtime generations are monotonic per anchor; a bump revokes prior-incarnation tokens before persistence commits. WI-3 stores the live generation in an owner-only locked file so same-PID Pi reload/new/resume/fork and Claude resume/clear transitions cannot inherit a VERIFIED lease.
|
||||
- Session IDs and cycle tokens use the OS cryptographic RNG. `Math.random` and model output are not token sources.
|
||||
- Framing and persistence failures fail closed. Sensitive tokens are not logged.
|
||||
- Built-in `0700`/`0600` filesystem modes provide same-principal hardening only, not socket authenticity against the same UID. WI-1 provides no distinct-principal isolation. That stronger deployment requires an external protected proxy, ACL, or service boundary, and the boundary must preserve authenticated client identity for the broker's `SO_PEERCRED` and ancestry authorization rather than substituting a shared proxy identity.
|
||||
- WI-2 whole-class authorization denies every consequential, unknown, and custom tool while UNVERIFIED; it does not inspect shell strings or trust wrapper selection. First-class Claude/Pi, both Claudex dispatch modes, PRDY, QA remediation, coord, orchestrator, and fleet starts converge on broker register-before-exec; Claudex additionally installs the mandatory all-tools hook inside its preserved isolated config and fails closed on unsafe settings.
|
||||
- The permanent `check-runtime-launches.py` suite/CI guard scans production source for direct literal, absolute-path, process-API, command-array, and dynamic Claude/Pi launches. It has no bypass allowlist: an unrecognized launch form fails CI until routed through the common boundary.
|
||||
- WI-2 promotion consumes a WI-1 cycle token before VERIFIED becomes visible. Observer revocation, runtime-generation replacement, broker restart, and monotonic TTL expiry remove authority.
|
||||
- WI-3 wires redundant Claude `PreCompact`/`SessionStart(compact)` and Pi `session_before_compact`/post-`session_compact` `context` observers to that same revoke action. If broker confirmation fails after an observer fires, the revoker advances the private generation as a local fence; subsequent authorization revokes the stale broker incarnation before any consequential allow.
|
||||
- Dual observer absence while a lease remains live is the named **bounded residual stale window**: consequential tools remain allowed until monotonic expiry, with no claimed within-window action bound. After expiry they are denied. Total observer-plus-gate absence remains T-C.
|
||||
- Receipt observation, payload construction, and constrained recovery implementation remain later surfaces. A receipt can become a promotion prerequisite but is never the safety mechanism.
|
||||
|
||||
## Named residual: promote-lease-lost-ACK (WI-3 D2-v5)
|
||||
|
||||
A valid `promote_lease` can leave a session `VERIFIED` in the broker while the client never learns of it. This is a named, bounded D2-v5 T-A residual — an **authority-observability divergence, not an authority divergence, not an ALLOW-risk, and not a retry double-apply**. It is disclosed here, not laundered.
|
||||
|
||||
**Window — where it can occur.** The broker commits token consumption and durable `VERIFIED` state _before_ the success reply becomes visible (see the promotion order in `lease-broker-protocol.md`). The residual is confined to the interval after that commit+fsync when the broker→client reply or peer-ACK is lost — for example an extreme-contention send failure or peer disconnect after `handle()` has already mutated and persisted state (the #838 fail-closed transport path). The lease mutation is already durable broker-side; only the acknowledgement to the client is lost. No uncommitted or partially-applied state is involved: the commit either happened (and is authoritative) or it did not (and no lease exists).
|
||||
|
||||
**Fail-safe direction — the client can only under-claim.** Broker intent is the ceiling; client authority is always ≤ broker intent, never more. Client-side authority-belief is granted only by a _received_ acknowledgement; a lost acknowledgement conveys nothing, so the client cannot conclude "verified" and continues to treat itself as `UNVERIFIED` (it re-verifies or recovers). If the client retries `promote_lease` with the same token, the token is already consumed and the broker rejects the retry (`PROMOTION_TOKEN_MISMATCH` / `INVALID_LEASE_TRANSITION`); there is no double-apply. The committed `VERIFIED` state the broker holds is authority the lease _legitimately earned_ from a real promotion — the broker authorizing consequential tools under it is correct, not inflation. Divergence is therefore strictly toward _less_ client authority than the broker granted; it never produces authority the broker did not grant.
|
||||
|
||||
**Bound — TTL plus the observer/gen-bump revoke backstop, self-healing.** The orphaned `VERIFIED` lease is indistinguishable to the broker from any other legitimately verified lease, so the identical D2-v5 revocation backstops dispose of it: any compaction observer (`PreCompact` / `SessionStart(compact)` for Claude; `session_before_compact` / post-`session_compact` `context` for Pi), any same-PID runtime-generation bump (reload/new/resume/fork), broker restart, or monotonic-time expiry returns the session to `UNVERIFIED`. Monotonic TTL expiry (capped at 300 seconds) is **unconditional** — it requires no observer at all — so the maximum exposure of the orphaned lease is one TTL, ≤ 300 s, after which the next consequential tool is denied with `LEASE_EXPIRED`. Any observer that fires shortens the window further. The residual self-heals: "≥1 observer fires OR expiry ⇒ revoke" catches the lost-ACK lease on the same terms as every other stale lease. As with the dual-observer-miss stale window, WI-3 makes no claim that the mutator gate bounds actions inside the residual interval; the interval is bounded by TTL and the revoke backstop, and the server-side branch-protection / required-CI / independent-review line remains the irreducible backstop for protected-repository mutations.
|
||||
|
||||
Coordinator security review must rerun the real socket/peercred and mutator-gate acceptance suites on an unrestricted Linux runner and obtain the mandated independent Opus-SECREV review before integration.
|
||||
@@ -0,0 +1,49 @@
|
||||
# Mos Runtime Portability M1 — Logical Identity and Fencing
|
||||
|
||||
## Boundary
|
||||
|
||||
M1 separates the logical Mosaic agent from any Claude, Pi, Codex, tmux, Matrix, or provider-native session. The normalized identity is:
|
||||
|
||||
```text
|
||||
(tenant_id, logical_agent_id, binding_id)
|
||||
```
|
||||
|
||||
`logical_agent_id` is a server-owned stable identifier. A connector is a replaceable holder of a lease for one binding; it is not the agent identity.
|
||||
|
||||
## Durable lease model
|
||||
|
||||
PostgreSQL table `logical_agent_connector_leases` has one unique row per identity/binding tuple. The current row records:
|
||||
|
||||
- an opaque lease UUID;
|
||||
- connector ID and normalized allowed scopes;
|
||||
- a positive decimal fencing epoch stored as PostgreSQL `bigint`;
|
||||
- acquired, heartbeat, expiry, release, and update timestamps.
|
||||
|
||||
Initial acquisition is insert-only. An existing active row causes `lease_held`. An expired or released row causes `takeover_required`; ordinary acquisition cannot recover it. Authorized takeover uses compare-and-swap against the expected epoch, rotates the lease UUID, and increments the epoch atomically. Heartbeat and release match the full identity, binding, connector, lease UUID, and epoch.
|
||||
|
||||
The companion `connector_lease_audit_log` is append-only metadata. It stores lifecycle event, outcome/reason, identity/binding/connector, epoch, correlation ID, and timestamp. It deliberately excludes scopes, grant objects, payloads, approval references, tokens, and credentials.
|
||||
|
||||
## Execution grants
|
||||
|
||||
`ConnectorLeaseCoordinator` issues a short-lived internal grant only after rereading the durable current lease. Defense-in-depth caps leases at 5 minutes and grants at 30 seconds by default; constructor options may tighten these limits. A grant is bound to tenant, logical agent, binding, connector, lease UUID, scope subset, expiry, and epoch.
|
||||
|
||||
Validation occurs immediately before adapter invocation and rereads PostgreSQL. The adapter receives only `ConnectorExecutionContext`; harness-native schemas remain behind the adapter. Validation denies:
|
||||
|
||||
- grants not minted by the current gateway process (including cloned/forged objects);
|
||||
- expired grants or leases;
|
||||
- released leases;
|
||||
- stale epochs or replaced connector/lease UUIDs;
|
||||
- missing/cross-tenant/cross-agent/cross-binding leases;
|
||||
- scopes not authorized by both grant and current lease.
|
||||
|
||||
A gateway restart intentionally invalidates process-local grants. The durable lease and epoch survive, and a fresh grant may be issued only after current-lease and gateway-policy validation.
|
||||
|
||||
## Concurrency and side-effect rule
|
||||
|
||||
The database CAS determines the sole current holder. A successful takeover makes every old-epoch validation fail. Connector adapters must consume and propagate the normalized lease epoch/context so downstream effect boundaries can also fence races that occur after gateway validation.
|
||||
|
||||
M1 does not provide exactly-once receipts or a side-effect journal. Those remain later #754 work; callers must not infer exactly-once delivery from lease fencing.
|
||||
|
||||
## Extension boundary
|
||||
|
||||
`ConnectorLeaseService` is the gateway-owned policy surface. Every policy decision receives the normalized requested scopes and TTL (or explicit `null` where no TTL applies), so a concrete policy can enforce least privilege and duration limits. Its production default policy denies every lease/grant operation until a server-configured connector policy is supplied. No M1 HTTP endpoint accepts caller-controlled tenant or logical identity, and no concrete Claude/Pi/Codex adapter or channel cutover is included.
|
||||
@@ -0,0 +1,72 @@
|
||||
# Whole mutator-class lease gate
|
||||
|
||||
WI-2 adds the framework-native authorization boundary for Claude (including the supported Claudex overlay) and Pi. Every runtime-reported tool name reaches the lease broker before execution. The gate classifies capabilities by the whole tool class; it never parses a Bash command to decide whether that particular string looks read-only.
|
||||
|
||||
## Default-deny policy
|
||||
|
||||
While a session is not VERIFIED, only these exact classes are allowed:
|
||||
|
||||
- Claude: `Read`, `Grep`, `Glob`, `Ls`, `Find`
|
||||
- Pi: `read`, `grep`, `find`, `ls`
|
||||
- Both runtimes: the fixed `mosaic_context_recover` primitive
|
||||
|
||||
Every other built-in, unknown tool, and custom/MCP tool is consequential by default and is denied. This includes Claude `Bash`, `Edit`, `Write`, and `NotebookEdit`, plus Pi `bash`, `edit`, and `write`. A compromised model therefore cannot bypass Mosaic wrappers by selecting raw `git`, `curl`, `kubectl`, provider, deployment, or filesystem commands inside a generic mutator—the generic mutator itself is blocked before its input executes.
|
||||
|
||||
## Broker-owned transition order
|
||||
|
||||
The authenticated broker is the sole lease writer:
|
||||
|
||||
1. `begin_verification` revokes existing authority and pending tokens first, then records `PENDING_VERIFICATION` and mints one WI-1 single-use promotion token bound to the exact cycle.
|
||||
2. `promote_lease` accepts only that session/generation/binding/token combination.
|
||||
3. Token consumption commits before the volatile lease becomes VERIFIED. Promotion is last and cannot be reached directly from UNVERIFIED.
|
||||
4. `revoke_lease`, a runtime-generation increase, broker restart, or monotonic expiry removes mutator authority.
|
||||
|
||||
The initial TTL is capped at the ratified 300-second maximum. A caller may request a shorter positive TTL but cannot lengthen the maximum. WI-3 installs the [compaction observer and generation lifecycle](compaction-revocation.md). Dual compaction-hook miss within an unexpired lease remains the ratified bounded T-A residual: consequential tools are allowed until expiry, with no claimed within-window action bound; once either observer revokes or TTL expires, the next consequential tool is denied.
|
||||
|
||||
A receipt is only a future promotion prerequisite. It is not an obedience, residency, or safety proof and never replaces this mechanical gate.
|
||||
|
||||
## Runtime adapters
|
||||
|
||||
`launch-runtime.py` registers itself with the broker and then `exec`s Claude or Pi so PID/starttime remain the authenticated parent anchor. It exports the broker-minted session ID and an owner-only generation-file reference to descendants; lifecycle hooks advance that file for same-PID replacement generations.
|
||||
|
||||
- Claude installs `mutator-gate.py` as an all-tools (`.*`) `PreToolUse` hook.
|
||||
- `mosaic claudex` and `mosaic yolo claudex` preserve their isolated `CLAUDE_CONFIG_DIR`, merge the mandatory hook into that isolated `settings.json`, and use the same register-before-exec launcher. Malformed or symlinked isolated settings deny launch.
|
||||
- Pi invokes the same executable from its `tool_call` handler.
|
||||
|
||||
The executable submits the runtime's actual tool name to `authorize_tool`. Missing identity, malformed input/reply, timeout, broker unavailability, or denial exits with status 2 and blocks fail-closed.
|
||||
|
||||
## Runtime-launch choke-point and permanent guard
|
||||
|
||||
Every repository-owned Claude/Pi launch entry converges on `launch-runtime.py`, either directly or through `mosaic` → `execLeaseGatedRuntime`. PRDY init/update and QA remediation invoke the wrapper directly so their existing prompts, dangerous-permission behavior, working directory, and environment survive without skipping broker registration. The raw Claude `--dangerously-skip-permissions` primitive is owned only by `launch-runtime.py`; callers request semantic `--dangerous` mode, and the wrapper validates Claude before injecting the primitive. `@mosaicstack/coord` rewrites direct Claude commands to `mosaic claude` and rejects unknown custom Claude launchers.
|
||||
|
||||
`check-runtime-launches.py` is the permanent completeness guard. It scans production shell, TypeScript/JavaScript, Python, and data launch definitions under `packages/`, `apps/`, `plugins/`, and `tools/`; direct literal, absolute-path, process-API, dynamic, command-substitution, `eval`, and variable-execution runtime launches fail. Shell comments are stripped with quote awareness, wrapper prefixes are tokenized with `shlex`, and only an invocation in command position with `--runtime` before the command separator is gated. Literal and tracked-variable command tokens use one terminal resolver after any nesting of `exec`, `command`, `nohup`, or `env` plus assignments. A direct command always wins over an inert marker on the same line. Independently, the raw dangerous primitive anywhere outside the choke-point is RED.
|
||||
|
||||
The command parser is a best-effort CI defense, not a complete shell interpreter. Alias/function redefinition, sourced commands, generated scripts, and encoded pipelines are intentionally residual rather than an invitation to chase an unbounded shell language. Two runtime controls backstop that residual surface: primitive ownership rejects a dangerous launch even when command identity is alias-indirected, and Claude's global `.*` `PreToolUse` hook invokes the broker gate for non-dangerous launches. Without `MOSAIC_LEASE_SESSION_ID`, representative read, mutator, and custom/MCP tools all fail closed with `GATE_UNAVAILABLE`. Hook absence or replacement remains in the documented T-C boundary.
|
||||
|
||||
### Parser stopping criterion
|
||||
|
||||
- **A — realistic parser matrix:** comments, inert strings/assignments, heredocs, continuations, chained commands, command substitution, `eval`, bare tracked variables, and quoted/unquoted tracked variables behind `exec`, `command`, `nohup`, or `env` are permanent RED regressions. Prefix-variable forms are covered in both multiline and same-line assignment shapes.
|
||||
- **B — residual backstops:** a dangerous alias-indirected launch is RED solely through primitive anchoring; a parser-missed non-dangerous alias launch is paired with an acceptance test proving the global all-tools hook denies every representative tool class as `GATE_UNAVAILABLE` without a lease.
|
||||
- **C — independent fresh review:** the parser class is considered complete only when reviewers find no new non-overlapping realistic evasion on the exact head. A and B are repository evidence; C is supplied by the fresh review round.
|
||||
|
||||
All three layers are load-bearing and complementary. The guard is mandatory in `@mosaicstack/mosaic`'s test script, so root CI fails on a future realistic bypass. Real-socket tests separately prove PRDY init/update and QA receive broker sessions and deny an unverified mutator.
|
||||
|
||||
The live inventory is emitted by:
|
||||
|
||||
```bash
|
||||
python3 packages/mosaic/framework/tools/lease-broker/check-runtime-launches.py --root . --json
|
||||
```
|
||||
|
||||
| Production launch family | Gated entries |
|
||||
| ------------------------------------------------------ | ------------: |
|
||||
| `@mosaicstack/coord` default/configured Claude command | 2 |
|
||||
| Fleet runtime start | 1 |
|
||||
| QA remediation + generated QA command | 2 |
|
||||
| Orchestrator command construction/session launches | 3 |
|
||||
| PRDY init/update | 2 |
|
||||
| Mosaic Claude/Pi/Claudex adapter and wrapper boundary | 4 |
|
||||
| **Total** | **14 / 14** |
|
||||
|
||||
## Assurance boundary
|
||||
|
||||
This closes T-A after an observer fires or lease expiry and T-B for in-runtime tool calls. Hook/extension absence, a runtime executing outside the gated launcher, ptrace/same-UID broker replacement, and other fully rotted behavior remain T-C. Server-side branch protection and required PR review/CI remain the irreducible line for protected repository mutations.
|
||||
@@ -0,0 +1,72 @@
|
||||
# Mission Manifest — CLI Unification & E2E First-Run
|
||||
|
||||
> Persistent document tracking full mission scope, status, and session history.
|
||||
> Updated by the orchestrator at each phase transition and milestone completion.
|
||||
|
||||
## Mission
|
||||
|
||||
**ID:** cli-unification-20260404
|
||||
**Statement:** Transform the Mosaic CLI from a partially-duplicated, manually-assembled experience into a single cohesive entry point that installs, configures, and controls the entire Mosaic system. Every Mosaic package gets first-class CLI surface. The first-run experience works end-to-end with no manual stitching. Gateway token recovery is possible without the web UI. Opt-in telemetry uses the published telemetry clients.
|
||||
**Phase:** Complete
|
||||
**Current Milestone:** —
|
||||
**Progress:** 8 / 8 milestones
|
||||
**Status:** completed
|
||||
**Last Updated:** 2026-04-05
|
||||
**Release:** [`mosaic-v0.0.24`](https://git.mosaicstack.dev/mosaicstack/mosaic-stack/releases/tag/mosaic-v0.0.24) (`@mosaicstack/[email protected]`, alpha — stays in 0.0.x until GA)
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [x] AC-1: Fresh machine `bash <(curl …install.sh)` → single command lands on a working authenticated gateway with a usable admin token; no secondary manual wizards required
|
||||
- [x] AC-2: `mosaic --help` lists every sub-package as a top-level command and is alphabetized for readability
|
||||
- [x] AC-3: `mosaic auth`, `mosaic brain`, `mosaic forge`, `mosaic log`, `mosaic macp`, `mosaic memory`, `mosaic queue`, `mosaic storage`, `mosaic telemetry` each expose at least one working subcommand that exercises the underlying package
|
||||
- [x] AC-4: Gateway admin token can be rotated or recovered from the CLI alone — operator is never stranded because the web UI is inaccessible
|
||||
- [x] AC-5: `mosaic telemetry` uses the published `@mosaicstack/telemetry-client-js` (from the Gitea npm registry); local OTEL stays for wide-event logging / post-mortems; remote upload is opt-in and disabled by default
|
||||
- [x] AC-6: Install → wizard → gateway install → TUI verification flow is a single cohesive path with clear state transitions and no dead ends
|
||||
- [x] AC-7: `@mosaicstack/mosaic` is the sole `mosaic` binary owner; `@mosaicstack/cli` is gone from the repo and all docs
|
||||
- [x] AC-8: All milestones ship as merged PRs with green CI, closed issues, and updated release notes
|
||||
|
||||
## Milestones
|
||||
|
||||
| # | ID | Name | Status | Branch | Issue | Started | Completed |
|
||||
| --- | ------ | ------------------------------------------------------------------------ | ------ | ----------------------------------- | --------------------------------- | ---------- | ---------- |
|
||||
| 1 | cu-m01 | Kill legacy @mosaicstack/cli package | done | chore/remove-cli-package-duplicate | #398 | 2026-04-04 | 2026-04-04 |
|
||||
| 2 | cu-m02 | Archive stale mission state + scaffold new mission | done | docs/mission-cli-unification | #399 | 2026-04-04 | 2026-04-04 |
|
||||
| 3 | cu-m03 | Fix gateway bootstrap token recovery (server + CLI paths) | done | feat/gateway-token-recovery | #411, #414 | 2026-04-05 | 2026-04-05 |
|
||||
| 4 | cu-m04 | Alphabetize + group `mosaic --help` output | done | feat/help-sort + feat/mosaic-config | #402, #408 | 2026-04-05 | 2026-04-05 |
|
||||
| 5 | cu-m05 | Sub-package CLI surface (auth/brain/forge/log/macp/memory/queue/storage) | done | feat/mosaic-\*-cli (x9) | #403–#407, #410, #412, #413, #415 | 2026-04-05 | 2026-04-05 |
|
||||
| 6 | cu-m06 | `mosaic telemetry` — local OTEL + opt-in remote upload | done | feat/mosaic-telemetry | #417 | 2026-04-05 | 2026-04-05 |
|
||||
| 7 | cu-m07 | Unified first-run UX (install.sh → wizard → gateway → TUI) | done | feat/mosaic-first-run-ux | #418 | 2026-04-05 | 2026-04-05 |
|
||||
| 8 | cu-m08 | Docs refresh + release tag | done | docs/cli-unification-release-v0.1.0 | #419 | 2026-04-05 | 2026-04-05 |
|
||||
|
||||
## Deployment
|
||||
|
||||
| Target | URL | Method |
|
||||
| -------------------- | --------- | ----------------------------------------------- |
|
||||
| Local tier (default) | localhost | `mosaic gateway install` — pglite + local queue |
|
||||
| Team tier | any host | `mosaic gateway install` — PG + Valkey |
|
||||
| Docker Compose (dev) | localhost | `docker compose up` for PG/Valkey/OTEL/Jaeger |
|
||||
|
||||
## Coordination
|
||||
|
||||
- **Primary Agent:** claude-opus-4-6[1m]
|
||||
- **Sibling Agents:** sonnet (standard implementation), haiku (status/explore/verify), codex (coding-heavy tasks)
|
||||
- **Shared Contracts:** `docs/PRD.md` (existing v0.1.0 PRD — still the long-term target), this manifest, `docs/TASKS.md`, `docs/scratchpads/cli-unification-20260404.md`
|
||||
|
||||
## Token Budget
|
||||
|
||||
| Metric | Value |
|
||||
| ------ | ------ |
|
||||
| Budget | TBD |
|
||||
| Used | ~80K |
|
||||
| Mode | normal |
|
||||
|
||||
## Session History
|
||||
|
||||
| Session | Runtime | Started | Duration | Ended Reason | Last Task |
|
||||
| ------- | --------------- | ---------- | -------- | ---------------- | ------------------------------------------------------------ |
|
||||
| 1 | claude-opus-4-6 | 2026-04-04 | ~4h | context-budget | cu-m01 + cu-m02 merged (#398, #399); open questions resolved |
|
||||
| 2 | claude-opus-4-6 | 2026-04-05 | ~6h | mission-complete | cu-m03..cu-m08 all merged; mosaic-v0.1.0 released |
|
||||
|
||||
## Scratchpad
|
||||
|
||||
Path: `docs/scratchpads/cli-unification-20260404.md`
|
||||
@@ -0,0 +1,90 @@
|
||||
# Tasks — CLI Unification & E2E First-Run
|
||||
|
||||
> Single-writer: orchestrator only. Workers read but never modify.
|
||||
>
|
||||
> **Mission:** cli-unification-20260404
|
||||
> **Schema:** `| id | status | description | issue | agent | branch | depends_on | estimate | notes |`
|
||||
> **Status values:** `not-started` | `in-progress` | `done` | `blocked` | `failed` | `needs-qa`
|
||||
> **Agent values:** `codex` | `sonnet` | `haiku` | `opus` | `glm-5` | `—` (auto)
|
||||
|
||||
## Milestone 1 — Kill legacy @mosaicstack/cli (done)
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ------ | ----------------------------------------------------------------- | ----- | ----- | ---------------------------------- | ---------- | -------- | --------------------------- |
|
||||
| CU-01-01 | done | Delete packages/cli directory; update workspace + docs references | #398 | opus | chore/remove-cli-package-duplicate | — | 5K | Merged c39433c3. 6685 LOC−. |
|
||||
|
||||
## Milestone 2 — Archive stale mission + scaffold new mission (done)
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ------ | ------------------------------------------------------------------ | ----- | ----- | ---------------------------- | ---------- | -------- | --------------------------------- |
|
||||
| CU-02-01 | done | Move stale MISSION-MANIFEST / TASKS / PRD-Harness to docs/archive/ | #399 | opus | docs/mission-cli-unification | CU-01-01 | 3K | Harness + storage missions done. |
|
||||
| CU-02-02 | done | Scaffold new MISSION-MANIFEST.md, TASKS.md, scratchpad | #399 | opus | docs/mission-cli-unification | CU-02-01 | 5K | This file + manifest + scratchpad |
|
||||
| CU-02-03 | done | PR review, merge, branch cleanup | #399 | opus | docs/mission-cli-unification | CU-02-02 | 2K | Merged as 6f15a84c |
|
||||
|
||||
## Milestone 3 — Gateway bootstrap token recovery
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ------ | ---------------------------------------------------------------------------------------------- | ----- | ------ | ------ | ---------- | -------- | ----------------------------- |
|
||||
| CU-03-01 | done | Implementation plan for BetterAuth-cookie recovery flow (decision locked 2026-04-04) | — | opus | — | CU-02-03 | 4K | Design locked; plan-only task |
|
||||
| CU-03-02 | done | Server: add recovery/rotate endpoint on apps/gateway/src/admin (gated by design from CU-03-01) | — | sonnet | — | CU-03-01 | 12K | |
|
||||
| CU-03-03 | done | CLI: `mosaic gateway login` — interactive BetterAuth sign-in, persist session | — | sonnet | — | CU-03-02 | 10K | |
|
||||
| CU-03-04 | done | CLI: `mosaic gateway config rotate-token` — mint new admin token via authenticated API | — | sonnet | — | CU-03-03 | 8K | |
|
||||
| CU-03-05 | done | CLI: `mosaic gateway config recover-token` — execute the recovery flow from CU-03-01 | — | sonnet | — | CU-03-03 | 10K | |
|
||||
| CU-03-06 | done | Install UX: fix the "user exists, no token" dead-end in runInstall bootstrapFirstUser path | — | sonnet | — | CU-03-05 | 8K | |
|
||||
| CU-03-07 | done | Tests: integration tests for each recovery path (happy + error) | — | sonnet | — | CU-03-06 | 10K | |
|
||||
| CU-03-08 | done | Code review + remediation | — | haiku | — | CU-03-07 | 4K | |
|
||||
|
||||
## Milestone 4 — `mosaic --help` alphabetize + grouping
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | ------ | ------ | ---------- | -------- | ------------------------------- |
|
||||
| CU-04-01 | done | Enable `configureHelp({ sortSubcommands: true })` on root program and each subgroup | — | sonnet | — | CU-02-03 | 3K | |
|
||||
| CU-04-02 | done | Group commands into sections (Runtime, Gateway, Framework, Platform) in help output | — | sonnet | — | CU-04-01 | 5K | |
|
||||
| CU-04-03 | done | Verify help snapshots render readably; update any docs with stale output | — | haiku | — | CU-04-02 | 3K | |
|
||||
| CU-04-04 | done | Top-level `mosaic config` command — `show`, `get <key>`, `set <key> <val>`, `edit`, `path` — wraps packages/mosaic/src/config/config-service.ts (framework/agent config; distinct from `mosaic gateway config`) | — | sonnet | — | CU-02-03 | 10K | New scope (decision 2026-04-04) |
|
||||
| CU-04-05 | done | Tests + code review for CU-04-04 | — | haiku | — | CU-04-04 | 4K | |
|
||||
|
||||
## Milestone 5 — Sub-package CLI surface
|
||||
|
||||
> Pattern: each sub-package exports `register<Name>Command(program: Command)` co-located with the library code (proven by `@mosaicstack/quality-rails`). Wire into `packages/mosaic/src/cli.ts`.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ------ | --------------------------------------------------------------------------------------------------------- | ----- | ------ | ------ | ---------- | -------- | ------------------- |
|
||||
| CU-05-01 | done | `mosaic forge` — subcommands: `run`, `status`, `resume`, `personas list` | — | sonnet | — | CU-02-03 | 18K | User priority |
|
||||
| CU-05-02 | done | `mosaic storage` — subcommands: `status`, `tier show`, `tier switch`, `export`, `import`, `migrate` | — | sonnet | — | CU-02-03 | 15K | |
|
||||
| CU-05-03 | done | `mosaic queue` — subcommands: `list`, `stats`, `pause/resume`, `jobs tail`, `drain` | — | sonnet | — | CU-02-03 | 12K | |
|
||||
| CU-05-04 | done | `mosaic memory` — subcommands: `search`, `stats`, `insights list`, `preferences list` | — | sonnet | — | CU-02-03 | 12K | |
|
||||
| CU-05-05 | done | `mosaic brain` — subcommands: `projects list/create`, `missions list`, `tasks list`, `conversations list` | — | sonnet | — | CU-02-03 | 15K | |
|
||||
| CU-05-06 | done | `mosaic auth` — subcommands: `users list/create/delete`, `sso list`, `sso test`, `sessions list` | — | sonnet | — | CU-03-03 | 15K | needs gateway login |
|
||||
| CU-05-07 | done | `mosaic log` — subcommands: `tail`, `search`, `export`, `level <level>` | — | sonnet | — | CU-02-03 | 10K | |
|
||||
| CU-05-08 | done | `mosaic macp` — subcommands: `tasks list`, `submit`, `gate`, `events tail` | — | sonnet | — | CU-02-03 | 12K | |
|
||||
| CU-05-09 | done | Wire all eight `register<Name>Command` calls into packages/mosaic/src/cli.ts | — | haiku | — | CU-05-01…8 | 3K | |
|
||||
| CU-05-10 | done | Integration test: `mosaic <cmd> --help` exits 0 for every new command | — | haiku | — | CU-05-09 | 5K | |
|
||||
|
||||
## Milestone 6 — `mosaic telemetry`
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ------ | ------------------------------------------------------------------------------------------------- | ----- | ------ | ------ | ---------- | -------- | ---------------------------------------------- |
|
||||
| CU-06-01 | done | Add `@mosaicstack/telemetry-client-js` as dependency of `@mosaicstack/mosaic` from Gitea registry | — | sonnet | — | CU-02-03 | 3K | |
|
||||
| CU-06-02 | done | `mosaic telemetry local` — status, tail, Jaeger link (wraps existing apps/gateway/src/tracing.ts) | — | sonnet | — | CU-06-01 | 8K | |
|
||||
| CU-06-03 | done | `mosaic telemetry` — status, opt-in, opt-out, test, upload (uses telemetry-client-js) | — | sonnet | — | CU-06-01 | 12K | Dry-run mode when server endpoint not yet live |
|
||||
| CU-06-04 | done | Persistent consent state in mosaic config; disabled by default | — | sonnet | — | CU-06-03 | 5K | |
|
||||
| CU-06-05 | done | Tests + code review | — | haiku | — | CU-06-04 | 5K | |
|
||||
|
||||
## Milestone 7 — Unified first-run UX
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ------ | ---------------------------------------------------------------------------------------------- | ----- | ------ | ------ | ---------- | -------- | ----- |
|
||||
| CU-07-01 | done | tools/install.sh: after npm install, hand off to `mosaic wizard` then `mosaic gateway install` | — | sonnet | — | CU-03-06 | 10K | |
|
||||
| CU-07-02 | done | `mosaic wizard` and `mosaic gateway install` coordination: shared state, no duplicate prompts | — | sonnet | — | CU-07-01 | 12K | |
|
||||
| CU-07-03 | done | Post-install verification step: "gateway healthy, tui connects, admin token on file" | — | sonnet | — | CU-07-02 | 8K | |
|
||||
| CU-07-04 | done | End-to-end test on a clean container from scratch | — | haiku | — | CU-07-03 | 8K | |
|
||||
|
||||
## Milestone 8 — Docs + release
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ------ | ---------------------------------------------------------------------- | ----- | ------ | ------ | ---------- | -------- | ----- |
|
||||
| CU-08-01 | done | Update README.md with new command tree, install flow, and feature list | — | sonnet | — | CU-07-04 | 8K | |
|
||||
| CU-08-02 | done | Update docs/guides/user-guide.md with all new sub-package commands | — | sonnet | — | CU-08-01 | 10K | |
|
||||
| CU-08-03 | done | Version bump `@mosaicstack/mosaic`, publish to Gitea registry | — | opus | — | CU-08-02 | 3K | |
|
||||
| CU-08-04 | done | Release notes, tag `v0.1.0-rc.N`, publish release on Gitea | — | opus | — | CU-08-03 | 3K | |
|
||||
@@ -0,0 +1,70 @@
|
||||
# Mission Manifest — Harness Foundation
|
||||
|
||||
> Persistent document tracking full mission scope, status, and session history.
|
||||
> Updated by the orchestrator at each phase transition and milestone completion.
|
||||
|
||||
## Mission
|
||||
|
||||
**ID:** harness-20260321
|
||||
**Statement:** Transform Mosaic Stack from a functional demo into a real multi-provider, task-routing AI harness. Persist all conversations, integrate frontier LLM providers (Anthropic, OpenAI, OpenRouter, Z.ai, Ollama), build granular task-aware agent routing, harden agent sessions, replace cron with BullMQ, and design the channel protocol for future Matrix/remote integration.
|
||||
**Phase:** Complete
|
||||
**Current Milestone:** All milestones done
|
||||
**Progress:** 7 / 7 milestones
|
||||
**Status:** complete
|
||||
**Last Updated:** 2026-03-22 UTC
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [x] AC-1: Send messages in TUI → restart TUI → resume conversation → agent has full history and context
|
||||
- [x] AC-2: Route a coding task to Claude Opus 4.6, a simple question to Haiku, a summarization to GLM-5 — all via granular routing rules
|
||||
- [x] AC-3: Two users exist, User A's memory searches never return User B's data
|
||||
- [x] AC-4: `/model claude-sonnet-4-6` in TUI switches the active model for subsequent messages
|
||||
- [x] AC-5: `/agent coding-agent` in TUI switches to a different agent with different system prompt and tools
|
||||
- [x] AC-6: BullMQ jobs execute on schedule, failures retry with backoff, admin can inspect via `/api/admin/jobs`
|
||||
- [x] AC-7: Channel protocol document exists with Matrix integration points defined, reviewed, and approved
|
||||
- [x] AC-8: Embeddings run on Ollama local models (no external API dependency for vector operations)
|
||||
- [x] AC-9: All five providers (Anthropic, OpenAI, OpenRouter, Z.ai, Ollama) connect, list models, and complete chat requests
|
||||
- [x] AC-10: Routing transparency — TUI displays which model was selected and the routing reason for each response
|
||||
|
||||
## Milestones
|
||||
|
||||
| # | ID | Name | Status | Branch | Issue | Started | Completed |
|
||||
| --- | ------ | ---------------------------------- | ------ | ------ | --------- | ---------- | ---------- |
|
||||
| 1 | ms-166 | Conversation Persistence & Context | done | — | #224–#231 | 2026-03-21 | 2026-03-21 |
|
||||
| 2 | ms-167 | Security & Isolation | done | — | #232–#239 | 2026-03-21 | 2026-03-21 |
|
||||
| 3 | ms-168 | Provider Integration | done | — | #240–#251 | 2026-03-21 | 2026-03-22 |
|
||||
| 4 | ms-169 | Agent Routing Engine | done | — | #252–#264 | 2026-03-22 | 2026-03-22 |
|
||||
| 5 | ms-170 | Agent Session Hardening | done | — | #265–#272 | 2026-03-22 | 2026-03-22 |
|
||||
| 6 | ms-171 | Job Queue Foundation | done | — | #273–#280 | 2026-03-22 | 2026-03-22 |
|
||||
| 7 | ms-172 | Channel Protocol Design | done | — | #281–#288 | 2026-03-22 | 2026-03-22 |
|
||||
|
||||
## Deployment
|
||||
|
||||
| Target | URL | Method |
|
||||
| -------------------- | --------- | -------------------------- |
|
||||
| Docker Compose (dev) | localhost | docker compose up |
|
||||
| Production | TBD | Docker Swarm via Portainer |
|
||||
|
||||
## Coordination
|
||||
|
||||
- **Primary Agent:** claude-opus-4-6
|
||||
- **Sibling Agents:** sonnet (workers), haiku (verification)
|
||||
- **Shared Contracts:** docs/PRD-Harness_Foundation.md, docs/TASKS.md
|
||||
|
||||
## Token Budget
|
||||
|
||||
| Metric | Value |
|
||||
| ------ | ------ |
|
||||
| Budget | — |
|
||||
| Used | ~2.5M |
|
||||
| Mode | normal |
|
||||
|
||||
## Session History
|
||||
|
||||
| Session | Runtime | Started | Duration | Ended Reason | Last Task |
|
||||
| ------- | --------------- | ---------- | -------- | ------------ | ----------------- |
|
||||
| 1 | claude-opus-4-6 | 2026-03-21 | ~6h | complete | M7-008 — all done |
|
||||
|
||||
## Scratchpad
|
||||
|
||||
Path: `docs/scratchpads/harness-20260321.md`
|
||||
@@ -0,0 +1,391 @@
|
||||
# PRD: Harness Foundation — Phase 9
|
||||
|
||||
## Metadata
|
||||
|
||||
- **Owner:** Jason Woltje
|
||||
- **Date:** 2026-03-21
|
||||
- **Status:** completed
|
||||
- **Phase:** 9 (post-MVP)
|
||||
- **Version Target:** v0.2.0
|
||||
- **Agent Harness:** [Pi SDK](https://github.com/badlogic/pi-mono)
|
||||
- **Best-Guess Mode:** true
|
||||
- **Repo:** `git.mosaicstack.dev/mosaic/mosaic-stack`
|
||||
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Mosaic Stack v0.1.0 delivered a functional skeleton — gateway boots, TUI connects, single-agent chat streams, basic auth works. But the system is not usable as a daily-driver harness:
|
||||
|
||||
1. **Chat messages are fire-and-forget.** The WebSocket gateway never calls ConversationsRepo. Context is lost on disconnect. Conversations can't be resumed with history. Cross-interface continuity (TUI → WebUI → Matrix) is impossible.
|
||||
|
||||
2. **Single provider (Ollama) with local models only.** No access to frontier models (Claude Opus 4.6, Codex gpt-5.4, GLM-5). The routing engine exists but has never been tested with real providers.
|
||||
|
||||
3. **No task-aware agent routing.** A coding task and a summarization task route to the same agent with the same model. There is no mechanism to match tasks to agents by capability, cost tier, or specialization.
|
||||
|
||||
4. **Memory is not user-scoped.** Insight vector search returns all users' data. Deploying multi-user is a security violation.
|
||||
|
||||
5. **Agent configs exist in DB but are ignored.** Stored system prompts, model preferences, and tool allowlists don't apply to sessions. The `/model` and `/agent` slash commands are stubbed.
|
||||
|
||||
6. **No job queue.** Background processing (summarization, GC, tier management) runs on fragile cron. No retry, no monitoring, no async task dispatch foundation for future agent orchestration.
|
||||
|
||||
7. **Plugin system is hollow.** Zero implementations. No defined message protocol. Blocks all remote interfaces (Matrix, Discord, Telegram) planned for Phase 10+.
|
||||
|
||||
**What this phase solves:** Transform Mosaic from a demo into a real multi-provider, task-routing AI harness that persists everything, routes intelligently, and is architecturally ready for multi-agent and remote control.
|
||||
|
||||
---
|
||||
|
||||
## Objectives
|
||||
|
||||
1. **Persistent conversations** — Every message saved, every conversation resumable, full context available across interfaces
|
||||
2. **Multi-provider LLM access** — Anthropic, OpenAI, OpenRouter, Z.ai, Ollama with proper auth flows
|
||||
3. **Task-aware agent routing** — Granular routing rules that match tasks to the right agent + model by capability, cost, and domain
|
||||
4. **Security isolation** — All data queries user-scoped, ready for multi-user deployment
|
||||
5. **Session hardening** — Agent configs apply, model/agent switching works mid-session
|
||||
6. **Reliable background processing** — BullMQ job queue replaces fragile cron
|
||||
7. **Channel protocol design** — Architecture for Matrix and remote interfaces, built into the foundation now
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
1. Conversation persistence — wire ChatGateway to ConversationsRepo, context loading on resume
|
||||
2. Multi-provider integration — Anthropic, OpenAI, OpenRouter, Z.ai, Ollama with auth flows
|
||||
3. Task-aware agent routing — granular routing rules with task classification and fallback chains
|
||||
4. Security isolation — user-scoped queries on all data paths (memory, conversations, agents)
|
||||
5. Agent session hardening — configs apply, model/agent switching, session resume
|
||||
6. Job queue — BullMQ replacing cron for background processing
|
||||
7. Channel protocol design — architecture document for Matrix and remote interfaces
|
||||
8. Embedding migration — Ollama-local embeddings replacing OpenAI dependency
|
||||
|
||||
### Out of Scope
|
||||
|
||||
1. Matrix homeserver deployment + appservice (Phase 10)
|
||||
2. Multi-agent orchestration / supervisor-worker pattern (Phase 10+)
|
||||
3. WebUI rebuild (future)
|
||||
4. Self-managing memory — compaction, merge, forget (future)
|
||||
5. Team workspace isolation (future)
|
||||
6. Remote channel plugins — WhatsApp, Discord, Telegram (Phase 10+, via Matrix)
|
||||
7. Fine-grained RBAC — project/agent/team roles (future)
|
||||
8. Agent-to-agent communication (Phase 10+)
|
||||
|
||||
## User/Stakeholder Requirements
|
||||
|
||||
1. As a user, I can resume a conversation after closing the TUI and the agent remembers the full context
|
||||
2. As a user, I can use frontier models (Claude Opus 4.6, Codex gpt-5.4) without manual provider configuration
|
||||
3. As a user, the system automatically selects the best model for my task (coding → powerful model, simple question → cheap model)
|
||||
4. As a user, I can override the automatic model selection with `/model <name>` at any time
|
||||
5. As a user, I can switch between specialized agents mid-session with `/agent <name>`
|
||||
6. As an admin, I can define routing rules that control which models handle which task types
|
||||
7. As an admin, I can monitor background job health and retry failed jobs
|
||||
8. As a user, my conversations, memories, and preferences are invisible to other users
|
||||
|
||||
## Functional Requirements
|
||||
|
||||
1. FR-1: ChatGateway persists every message (user, assistant, tool call, thinking) to the conversations/messages tables
|
||||
2. FR-2: On session resume with an existing conversationId, message history is loaded from DB and injected into the agent session context
|
||||
3. FR-3: When conversation history exceeds 80% of the model's context window, older messages are summarized and prepended as a context checkpoint
|
||||
4. FR-4: Five LLM providers are registered with the gateway: Anthropic (Claude Sonnet 4.6, Opus 4.6, Haiku 4.5), OpenAI (Codex gpt-5.4), OpenRouter (dynamic model list), Z.ai (GLM-5), Ollama (local models)
|
||||
5. FR-5: Each provider supports API key auth; Anthropic and OpenAI additionally support OAuth (URL-display + callback pattern)
|
||||
6. FR-6: Provider credentials are stored per-user in the DB (encrypted), not in environment variables
|
||||
7. FR-7: A routing engine classifies each user message by taskType, complexity, domain, and required capabilities, then selects the optimal provider/model via priority-ordered rules
|
||||
8. FR-8: Default routing rules are seeded on first run; admins can customize system-wide rules; users can set per-session overrides
|
||||
9. FR-9: Routing decisions are transparent — the TUI shows which model was selected and why
|
||||
10. FR-10: Agent configs (system prompt, default model, tool allowlist, skills) stored in DB are applied when creating agent sessions
|
||||
11. FR-11: `/model <name>` switches the active model for subsequent messages in the current session
|
||||
12. FR-12: `/agent <name>` switches to a different agent config, loading its system prompt, tools, and default model
|
||||
13. FR-13: All memory queries (insight vector search, preferences) filter by userId
|
||||
14. FR-14: BullMQ handles background jobs (summarization, GC, tier management) with retry, backoff, and monitoring
|
||||
15. FR-15: Embeddings are served locally via Ollama (nomic-embed-text or mxbai-embed-large) with no external API dependency
|
||||
|
||||
## Non-Functional Requirements
|
||||
|
||||
1. **Security:** All data queries include userId filter. Provider credentials encrypted at rest. No cross-user data leakage. OAuth tokens stored securely with refresh handling.
|
||||
2. **Performance:** Message persistence adds <50ms to message relay latency. Routing classification <100ms per message. Provider health checks run on configurable interval (default 60s) without blocking requests.
|
||||
3. **Reliability:** BullMQ jobs retry with exponential backoff (3 attempts default). Provider failover: if primary provider is unhealthy, fallback chain activates automatically. Conversation context survives TUI restart.
|
||||
4. **Observability:** Routing decisions logged with classification details. Job execution logged to agent_logs. Provider health status exposed via `/api/providers/health`. Session metrics (tokens, model switches, duration) persisted in DB.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] AC-1: Send messages in TUI → restart TUI → resume conversation → agent has full history and context
|
||||
- [ ] AC-2: Route a coding task to Claude Opus 4.6, a simple question to Haiku, a summarization to GLM-5 — all via granular routing rules
|
||||
- [ ] AC-3: Two users exist, User A's memory searches never return User B's data
|
||||
- [ ] AC-4: `/model claude-sonnet-4-6` in TUI switches the active model for subsequent messages
|
||||
- [ ] AC-5: `/agent coding-agent` in TUI switches to a different agent with different system prompt and tools
|
||||
- [ ] AC-6: BullMQ jobs execute on schedule, failures retry with backoff, admin can inspect via `/api/admin/jobs`
|
||||
- [ ] AC-7: Channel protocol document exists with Matrix integration points defined, reviewed, and approved
|
||||
- [ ] AC-8: Embeddings run on Ollama local models (no external API dependency for vector operations)
|
||||
- [ ] AC-9: All five providers (Anthropic, OpenAI, OpenRouter, Z.ai, Ollama) connect, list models, and complete chat requests
|
||||
- [ ] AC-10: Routing transparency — TUI displays which model was selected and the routing reason for each response
|
||||
|
||||
## Testing and Verification Expectations
|
||||
|
||||
1. **Baseline checks:** `pnpm typecheck`, `pnpm lint`, `pnpm format:check` — all green before any push
|
||||
2. **Unit tests:** Routing engine rules matching, task classifier, provider adapter registration, message persistence
|
||||
3. **Integration tests:** Two-user isolation (M2-007), provider round-trip (M3-012), routing end-to-end (M4-013), session resume with context (M1-008)
|
||||
4. **Situational tests per milestone:** Each milestone has a verify task that exercises the delivered functionality end-to-end
|
||||
5. **Evidence format:** Test output + manual verification notes in scratchpad per milestone
|
||||
|
||||
## Constraints and Dependencies
|
||||
|
||||
| Type | Item | Notes |
|
||||
| ---------- | ------------------------------- | -------------------------------------------------------------------------------------- |
|
||||
| Dependency | `@anthropic-ai/sdk` | npm, required for M3-002 |
|
||||
| Dependency | `openai` | npm, required for M3-003 |
|
||||
| Dependency | `bullmq` | npm, Valkey-compatible, required for M6 |
|
||||
| Dependency | Ollama embedding models | `ollama pull nomic-embed-text`, required for M3-009 |
|
||||
| Dependency | Pi SDK provider adapter support | ASSUMPTION: supported — verify in M3-001 |
|
||||
| External | Anthropic OAuth credentials | Requires Anthropic Console setup |
|
||||
| External | OpenAI OAuth credentials | Requires OpenAI Platform setup |
|
||||
| External | Z.ai API key | Requires Z.ai account |
|
||||
| External | OpenRouter API key | Requires OpenRouter account |
|
||||
| Constraint | Valkey 8 compatibility | BullMQ requires Redis 6+; Valkey 8 is compatible |
|
||||
| Constraint | Embedding dimension migration | Switching from 1536 (OpenAI) to 768/1024 (Ollama) requires re-embedding or fresh start |
|
||||
|
||||
---
|
||||
|
||||
## Assumptions
|
||||
|
||||
1. ASSUMPTION: Pi SDK supports custom provider adapters for all target LLM providers. If not, adapters wrap native SDKs behind Pi's interface. **Rationale:** Gateway already uses Pi with Ollama via a custom adapter pattern.
|
||||
2. ASSUMPTION: BullMQ is Valkey-compatible. **Rationale:** BullMQ documents Redis 6+ compatibility; Valkey 8 is Redis-compatible.
|
||||
3. ASSUMPTION: Ollama can serve embedding models (nomic-embed-text, mxbai-embed-large) with acceptable quality. **Rationale:** Ollama supports embedding endpoints natively.
|
||||
4. ASSUMPTION: Anthropic and OpenAI OAuth flows can be handled via URL-display + token callback pattern (same as existing provider auth). **Rationale:** Both providers offer standard OAuth 2.0 flows.
|
||||
5. ASSUMPTION: Z.ai GLM-5 uses an API format compatible with OpenAI or has a documented SDK. **Rationale:** Most LLM providers converge on OpenAI-compatible APIs.
|
||||
6. ASSUMPTION: The existing Pi SDK session model supports mid-session model switching without destroying session state. If not, we destroy and recreate with conversation history. **Rationale:** Acceptable fallback — context is persisted in DB.
|
||||
7. ASSUMPTION: Channel protocol design can be completed without a running Matrix homeserver. **Rationale:** Matrix protocol is well-documented; design is architecture, not integration.
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
### Milestone 1: Conversation Persistence & Context
|
||||
|
||||
**Goal:** Every message persisted. Every conversation resumable with full context.
|
||||
|
||||
| Task | Description |
|
||||
| ------ | ------------------------------------------------------------------------------------------------------------ |
|
||||
| M1-001 | Wire ChatGateway.handleMessage() → ConversationsRepo.addMessage() for user messages |
|
||||
| M1-002 | Wire agent event relay → ConversationsRepo.addMessage() for assistant responses (text, tool calls, thinking) |
|
||||
| M1-003 | Store message metadata: model used, provider, token counts, tool call details, timestamps |
|
||||
| M1-004 | On session resume (existing conversationId), load message history from DB and inject into Pi session context |
|
||||
| M1-005 | Context window management: if history exceeds model context, summarize older messages and prepend summary |
|
||||
| M1-006 | Conversation search: full-text search on messages table via `/api/conversations/search` |
|
||||
| M1-007 | TUI: `/history` command to display conversation message count and context usage |
|
||||
| M1-008 | Verify: send messages → kill TUI → resume with `-c <id>` → agent references prior context |
|
||||
|
||||
### Milestone 2: Security & Isolation
|
||||
|
||||
**Goal:** All data queries user-scoped. Safe for multi-user deployment.
|
||||
|
||||
| Task | Description |
|
||||
| ------ | --------------------------------------------------------------------------------------------------------------- |
|
||||
| M2-001 | Audit InsightsRepo: add `userId` filter to `searchByEmbedding()` vector search |
|
||||
| M2-002 | Audit InsightsRepo: add `userId` filter to `findByUser()`, `decayOldInsights()` |
|
||||
| M2-003 | Audit PreferencesRepo: verify all queries filter by userId |
|
||||
| M2-004 | Audit agent memory tools: verify `memory_search`, `memory_save_*`, `memory_get_*` all scope to session user |
|
||||
| M2-005 | Audit ConversationsRepo: verify ownership check on findById, update, delete, addMessage, findMessages |
|
||||
| M2-006 | Audit AgentsRepo: verify `findAccessible()` returns only user's agents + system agents |
|
||||
| M2-007 | Add integration test: create two users, populate data for each, verify cross-user isolation on every query path |
|
||||
| M2-008 | Audit Valkey keys: verify session keys include userId or are not enumerable across users |
|
||||
|
||||
### Milestone 3: Provider Integration
|
||||
|
||||
**Goal:** Five providers operational with proper auth, health checking, and capability metadata.
|
||||
|
||||
| Task | Description |
|
||||
| ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| M3-001 | Refactor ProviderService into provider adapter pattern: `IProviderAdapter` interface with `register()`, `listModels()`, `healthCheck()`, `createClient()` |
|
||||
| M3-002 | Anthropic adapter: `@anthropic-ai/sdk`, register Claude Sonnet 4.6 + Opus 4.6, OAuth flow (URL display + callback), API key fallback |
|
||||
| M3-003 | OpenAI adapter: `openai` SDK, register Codex gpt-5.4, OAuth flow, API key fallback |
|
||||
| M3-004 | OpenRouter adapter: OpenAI-compatible client, API key auth, dynamic model list from `/api/v1/models` |
|
||||
| M3-005 | Z.ai GLM adapter: register GLM-5, API key auth, research and implement API format |
|
||||
| M3-006 | Ollama adapter: refactor existing Ollama integration into adapter pattern, add embedding model support |
|
||||
| M3-007 | Provider health check: periodic probe (configurable interval), status per provider, expose via `/api/providers/health` |
|
||||
| M3-008 | Model capability matrix: define per-model metadata (tier, context window, tool support, vision, streaming, embedding capable) |
|
||||
| M3-009 | Refactor EmbeddingService: replace OpenAI-hardcoded client with provider-agnostic interface, Ollama as default (nomic-embed-text or mxbai-embed-large) |
|
||||
| M3-010 | OAuth token storage: persist provider tokens per user in DB (encrypted), refresh flow |
|
||||
| M3-011 | Provider config UI support: `/api/providers` CRUD for user-scoped provider credentials |
|
||||
| M3-012 | Verify: each provider connects, lists models, completes a chat request, handles errors gracefully |
|
||||
|
||||
### Milestone 4: Agent Routing Engine
|
||||
|
||||
**Goal:** Granular, rule-based routing that matches tasks to the right agent and model by capability, cost, and domain specialization.
|
||||
|
||||
| Task | Description |
|
||||
| ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| M4-001 | Define routing rule schema: `RoutingRule { name, priority, conditions[], action }` stored in DB |
|
||||
| M4-002 | Condition types: `taskType` (coding, research, summarization, conversation, analysis, creative), `complexity` (simple, moderate, complex), `domain` (frontend, backend, devops, docs, general), `costTier` (cheap, standard, premium), `requiredCapabilities` (tools, vision, long-context, reasoning) |
|
||||
| M4-003 | Action types: `routeTo { provider, model, agentConfigId?, systemPromptOverride?, toolAllowlist? }` |
|
||||
| M4-004 | Default routing rules (seed data): coding → Opus 4.6, simple Q&A → Sonnet 4.6, summarization → GLM-5, research → Codex gpt-5.4, local/offline → Ollama llama3.2 |
|
||||
| M4-005 | Task classification: lightweight classifier that infers taskType + complexity from user message (can be rule-based regex/keyword initially, LLM-assisted later) |
|
||||
| M4-006 | Routing decision pipeline: classify task → match rules by priority → select best available provider/model → fallback chain if primary unavailable |
|
||||
| M4-007 | Routing override: user can force a specific model via `/model <name>` regardless of routing rules |
|
||||
| M4-008 | Routing transparency: include routing decision in `session:info` event (why this model was selected) |
|
||||
| M4-009 | Routing rules CRUD: `/api/routing/rules` — list, create, update, delete, reorder priority |
|
||||
| M4-010 | Per-user routing overrides: users can customize default rules for their sessions |
|
||||
| M4-011 | Agent specialization: agents can declare capabilities in their config (domains, preferred models, tool sets) |
|
||||
| M4-012 | Routing integration: wire routing engine into ChatGateway — every new message triggers routing decision before agent dispatch |
|
||||
| M4-013 | Verify: send a coding question → routed to Opus; send "summarize this" → routed to GLM-5; send "what time is it" → routed to cheap tier |
|
||||
|
||||
### Milestone 5: Agent Session Hardening
|
||||
|
||||
**Goal:** Agent configs apply to sessions. Model and agent switching work mid-session.
|
||||
|
||||
| Task | Description |
|
||||
| ------ | ------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| M5-001 | Wire ChatGateway: on session create, load agent config from DB (system prompt, model, provider, tool allowlist, skills) |
|
||||
| M5-002 | `/model <name>` command: end-to-end wiring — TUI → socket `command:execute` → gateway switches provider/model → new messages use new model |
|
||||
| M5-003 | `/agent <name>` command: switch to different agent config mid-session — loads new system prompt, tools, and default model |
|
||||
| M5-004 | Session ↔ conversation binding: persist sessionId on conversation record, allow session resume via conversation ID |
|
||||
| M5-005 | Session info broadcast: on model/agent switch, emit `session:info` with updated provider, model, agent name |
|
||||
| M5-006 | Agent creation from TUI: `/agent new` command creates agent config via gateway API |
|
||||
| M5-007 | Session metrics: track per-session token usage, model switches, duration — persist in DB |
|
||||
| M5-008 | Verify: start TUI → `/model claude-opus-4-6` → verify response uses Opus → `/agent research-bot` → verify system prompt changes |
|
||||
|
||||
### Milestone 6: Job Queue Foundation
|
||||
|
||||
**Goal:** Reliable background processing via BullMQ. Foundation for future agent task orchestration.
|
||||
|
||||
| Task | Description |
|
||||
| ------ | ------------------------------------------------------------------------------------------------------------ |
|
||||
| M6-001 | Add BullMQ dependency, configure with Valkey connection |
|
||||
| M6-002 | Create queue service: typed job definitions, worker registration, error handling with exponential backoff |
|
||||
| M6-003 | Migrate summarization cron → BullMQ repeatable job |
|
||||
| M6-004 | Migrate GC (session cleanup) → BullMQ repeatable job |
|
||||
| M6-005 | Migrate tier management (log archival) → BullMQ repeatable job |
|
||||
| M6-006 | Admin jobs API: `GET /api/admin/jobs` — list active/completed/failed jobs, retry failed, pause/resume queues |
|
||||
| M6-007 | Job event logging: emit job start/complete/fail events to agent_logs for observability |
|
||||
| M6-008 | Verify: jobs execute on schedule, deliberate failure retries with backoff, admin endpoint shows job history |
|
||||
|
||||
### Milestone 7: Channel Protocol Design
|
||||
|
||||
**Goal:** Architecture document defining how remote interfaces (Matrix, Discord, Telegram) will integrate. No code — design only. Built into foundation now so Phase 10+ doesn't require gateway rewrites.
|
||||
|
||||
| Task | Description |
|
||||
| ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| M7-001 | Define `IChannelAdapter` interface: lifecycle (connect, disconnect, health), message flow (receiveMessage → gateway, sendMessage ← gateway), identity mapping (channel user ↔ Mosaic user) |
|
||||
| M7-002 | Define channel message protocol: canonical message format that all adapters translate to/from (content, metadata, attachments, thread context) |
|
||||
| M7-003 | Design Matrix integration: appservice registration, room ↔ conversation mapping, space ↔ team mapping, agent ghost users, power levels for human observation |
|
||||
| M7-004 | Design conversation multiplexing: same conversation accessible from TUI + WebUI + Matrix simultaneously, real-time sync via gateway events |
|
||||
| M7-005 | Design remote auth bridging: how a Matrix/Discord message authenticates to Mosaic (token linking, OAuth bridge, invite-based provisioning) |
|
||||
| M7-006 | Design agent-to-agent communication via Matrix rooms: room per agent pair, human can join to observe, message format for structured agent dialogue |
|
||||
| M7-007 | Design multi-user isolation in Matrix: space-per-team, room visibility rules, encryption considerations, admin visibility |
|
||||
| M7-008 | Publish architecture doc: `docs/architecture/channel-protocol.md` — reviewed and approved before Phase 10 |
|
||||
|
||||
---
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### Pi SDK Provider Adapter Pattern
|
||||
|
||||
The agent layer stays on Pi SDK. Provider diversity is solved at the adapter layer below Pi:
|
||||
|
||||
```
|
||||
Provider SDKs (@anthropic-ai/sdk, openai, etc.)
|
||||
→ IProviderAdapter implementations
|
||||
→ ProviderRegistry (Pi SDK compatible)
|
||||
→ Agent Session (Pi SDK) — tool loops, streaming, context
|
||||
→ AgentService — lifecycle, routing, events
|
||||
→ ChatGateway — WebSocket to all interfaces
|
||||
```
|
||||
|
||||
Adding a provider means implementing `IProviderAdapter`. Everything above stays unchanged.
|
||||
|
||||
### Routing Decision Flow
|
||||
|
||||
```
|
||||
User sends message
|
||||
→ Task classifier (regex/keyword, optionally LLM-assisted)
|
||||
→ { taskType, complexity, domain, requiredCapabilities }
|
||||
→ RoutingEngine.resolve(classification, userOverrides, availableProviders)
|
||||
→ Match rules by priority
|
||||
→ Check provider health
|
||||
→ Apply fallback chain
|
||||
→ Return { provider, model, agentConfigId }
|
||||
→ AgentService.createOrResumeSession(routingResult)
|
||||
→ Session uses selected provider/model
|
||||
→ Emit session:info with routing decision explanation
|
||||
```
|
||||
|
||||
### Embedding Strategy
|
||||
|
||||
Replace OpenAI-hardcoded embedding service with provider-agnostic interface:
|
||||
|
||||
- **Default:** Ollama serving `nomic-embed-text` (768-dim) or `mxbai-embed-large` (1024-dim)
|
||||
- **Fallback:** Any OpenAI-compatible embedding API
|
||||
- **Migration:** Update pgvector column dimension if switching from 1536 (OpenAI) to 768/1024 (Ollama models)
|
||||
- **No external API dependency** for vector operations in default configuration
|
||||
|
||||
### Context Window Management
|
||||
|
||||
When conversation history exceeds model context:
|
||||
|
||||
1. Calculate token count of full history
|
||||
2. If exceeds 80% of model context window, trigger summarization
|
||||
3. Summarize oldest N messages into a condensed context block
|
||||
4. Prepend summary + keep recent messages within context budget
|
||||
5. Store summary as a "context checkpoint" message in DB
|
||||
|
||||
### Model Reference
|
||||
|
||||
| Provider | Model | Tier | Context | Tools | Vision | Embedding |
|
||||
| ---------- | ----------------- | ---------- | ------- | ------ | ------ | -------------- |
|
||||
| Anthropic | Claude Opus 4.6 | premium | 200K | yes | yes | no |
|
||||
| Anthropic | Claude Sonnet 4.6 | standard | 200K | yes | yes | no |
|
||||
| Anthropic | Claude Haiku 4.5 | cheap | 200K | yes | yes | no |
|
||||
| OpenAI | Codex gpt-5.4 | premium | 128K+ | yes | yes | no |
|
||||
| Z.ai | GLM-5 | standard | TBD | TBD | TBD | no |
|
||||
| OpenRouter | varies | varies | varies | varies | varies | no |
|
||||
| Ollama | llama3.2 | local/free | 128K | yes | no | no |
|
||||
| Ollama | nomic-embed-text | — | — | — | — | yes (768-dim) |
|
||||
| Ollama | mxbai-embed-large | — | — | — | — | yes (1024-dim) |
|
||||
|
||||
### Default Routing Rules (Seed Data)
|
||||
|
||||
| Priority | Condition | Route To |
|
||||
| -------- | ------------------------------------------------------------- | ------------- |
|
||||
| 1 | taskType=coding AND complexity=complex | Opus 4.6 |
|
||||
| 2 | taskType=coding AND complexity=moderate | Sonnet 4.6 |
|
||||
| 3 | taskType=coding AND complexity=simple | Codex gpt-5.4 |
|
||||
| 4 | taskType=research | Codex gpt-5.4 |
|
||||
| 5 | taskType=summarization | GLM-5 |
|
||||
| 6 | taskType=analysis AND requiredCapabilities includes reasoning | Opus 4.6 |
|
||||
| 7 | taskType=conversation | Sonnet 4.6 |
|
||||
| 8 | taskType=creative | Sonnet 4.6 |
|
||||
| 9 | costTier=cheap OR domain=general | Haiku 4.5 |
|
||||
| 10 | fallback (no rule matched) | Sonnet 4.6 |
|
||||
| 99 | provider=ollama forced OR offline mode | llama3.2 |
|
||||
|
||||
Rules are user-customizable. Admins set system defaults; users override for their sessions.
|
||||
|
||||
---
|
||||
|
||||
## Risks and Open Questions
|
||||
|
||||
| Risk | Impact | Mitigation |
|
||||
| ------------------------------------------------------- | ------------------------- | ----------------------------------------------------------------------------------------------------------------- |
|
||||
| Pi SDK doesn't support custom provider adapters cleanly | High — blocks M3 | Verify in M3-001; fallback: wrap native SDKs and bypass Pi's registry, feeding responses into Pi's session format |
|
||||
| BullMQ + Valkey incompatibility | Medium — blocks M6 | Test in M6-001 before migrating jobs; fallback: use `bullmq` with `ioredis` directly |
|
||||
| Embedding dimension migration (1536 → 768/1024) | Medium — data migration | Run migration script to re-embed existing insights; or start fresh if insight count is low |
|
||||
| Z.ai GLM-5 API undocumented | Low — blocks one provider | Deprioritize; other 4 providers cover all use cases |
|
||||
| Context window summarization quality | Medium — affects UX | Start with simple truncation; add LLM summarization iteratively |
|
||||
| OAuth flow complexity in TUI (no browser redirect) | Medium | URL-display + clipboard + Valkey poll token pattern (already designed in P8-012) |
|
||||
|
||||
### Open Questions
|
||||
|
||||
1. What is the Z.ai GLM-5 API format? OpenAI-compatible or custom SDK? (Research in M3-005)
|
||||
2. Should routing classification use LLM-assisted classification from the start, or rule-based only? (ASSUMPTION: rule-based first, LLM-assisted later)
|
||||
3. What Ollama embedding model provides the best quality/performance tradeoff? (Test nomic-embed-text vs mxbai-embed-large in M3-009)
|
||||
4. Should provider credentials be stored in DB per-user, or remain environment-variable based for system-wide providers? (ASSUMPTION: hybrid — env vars for system defaults, DB for per-user overrides)
|
||||
|
||||
---
|
||||
|
||||
## Milestone / Delivery Intent
|
||||
|
||||
1. **Target version:** v0.2.0
|
||||
2. **Milestone count:** 7
|
||||
3. **Definition of done:** All 10 acceptance criteria verified with evidence, all quality gates green, PRD status updated to `completed`
|
||||
4. **Delivery order:** M1 (persistence) → M2 (security) → M3 (providers) → M4 (routing) → M5 (sessions) → M6 (jobs) → M7 (channel design)
|
||||
5. **M1 and M2 are prerequisites** — no provider or routing work begins until conversations persist and data is user-scoped
|
||||
+57
@@ -0,0 +1,57 @@
|
||||
# Mission Manifest — Install UX Hardening
|
||||
|
||||
> Persistent document tracking full mission scope, status, and session history.
|
||||
> Updated by the orchestrator at each phase transition and milestone completion.
|
||||
|
||||
## Mission
|
||||
|
||||
**ID:** install-ux-hardening-20260405
|
||||
**Statement:** Close the remaining gaps in the Mosaic Stack first-run and teardown experience uncovered by the post-`cli-unification` audit. A user MUST be able to cleanly uninstall the stack; the wizard MUST make security-sensitive surfaces visible (hooks, password entry); and CI/headless installs MUST NOT hang on interactive prompts. The longer-term goal is a single cohesive first-run flow that collapses `mosaic wizard` and `mosaic gateway install` into one state-bridged experience.
|
||||
**Phase:** Complete
|
||||
**Current Milestone:** —
|
||||
**Progress:** 3 / 3 milestones
|
||||
**Status:** complete
|
||||
**Last Updated:** 2026-04-05 (mission complete)
|
||||
**Parent Mission:** [cli-unification-20260404](./archive/missions/cli-unification-20260404/MISSION-MANIFEST.md) (complete)
|
||||
|
||||
## Context
|
||||
|
||||
Post-merge audit of `cli-unification-20260404` (AC-1, AC-6) validated that the first-run wizard covers first user, password, admin tokens, gateway instance config, skills, and SOUL.md/USER.md init. The audit surfaced six gaps, grouped into three tracks of independent value.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [x] AC-1: `mosaic uninstall` (top-level) cleanly reverses every mutation made by `tools/install.sh` — framework data, npm CLI, nested stack deps, runtime asset injections in `~/.claude/`, npmrc scope mapping, PATH edits. Dry-run supported. `--keep-data` preserves memory + user files + gateway DB. (PR #429)
|
||||
- [x] AC-2: `curl … | bash -s -- --uninstall` works without requiring a functioning CLI. (PR #429)
|
||||
- [x] AC-3: Password entry in `bootstrapFirstUser` is masked (no plaintext echo); confirm prompt added. (PR #431)
|
||||
- [x] AC-4: Wizard has an explicit hooks stage that previews which hooks will be installed, asks for confirmation, and records the user's choice. `mosaic config hooks list|enable|disable` surface exists. (PR #431 — consent; PR #433 — finalize-stage gating now honors `state.hooks.accepted === false` end-to-end)
|
||||
- [x] AC-5: `runConfigWizard` and `bootstrapFirstUser` accept a headless path (env vars + `--yes`) so `tools/install.sh --yes` + `MOSAIC_ASSUME_YES=1` completes end-to-end in CI without TTY. (PR #431)
|
||||
- [x] AC-6: `mosaic wizard` and `mosaic gateway install` are collapsed into a single cohesive entry point with shared state; gateway install is now terminal stages 11 & 12 of `runWizard`, session-file bridge removed, `mosaic gateway install` preserved as a thin standalone wrapper. (PR #433)
|
||||
- [x] AC-7: All milestones shipped as merged PRs with green CI and closed issues. (PRs #429, #431, #433)
|
||||
|
||||
## Milestones
|
||||
|
||||
| # | ID | Name | Status | Branch | Issue | Started | Completed |
|
||||
| --- | ------- | --------------------------------------------------------- | ------ | ----------------------- | ----- | ---------- | ---------- |
|
||||
| 1 | IUH-M01 | `mosaic uninstall` — top-level teardown + shell wrapper | done | feat/mosaic-uninstall | #425 | 2026-04-05 | 2026-04-05 |
|
||||
| 2 | IUH-M02 | Wizard remediation — hooks visibility, pwd mask, headless | done | feat/wizard-remediation | #426 | 2026-04-05 | 2026-04-05 |
|
||||
| 3 | IUH-M03 | Unified first-run wizard (collapse wizard + gateway) | done | feat/unified-first-run | #427 | 2026-04-05 | 2026-04-05 |
|
||||
|
||||
## Subagent Delegation Plan
|
||||
|
||||
| Milestone | Recommended Tier | Rationale |
|
||||
| --------- | ---------------- | ---------------------------------------------------------------------- |
|
||||
| IUH-M01 | sonnet | Standard feature work — new command surface mirroring existing install |
|
||||
| IUH-M02 | sonnet | Small surgical fixes across 3-4 files |
|
||||
| IUH-M03 | opus | Architectural refactor; state machine design decisions |
|
||||
|
||||
## Risks
|
||||
|
||||
- **Reversal completeness** — runtime asset linking creates `.mosaic-bak-*` backups; uninstall must honor them vs. when to delete. Ambiguity without an install manifest.
|
||||
- **npm global nested deps** — `npm uninstall -g @mosaicstack/mosaic` removes nested `@mosaicstack/*`, but ownership conflicts with explicitly installed peer packages (`@mosaicstack/gateway`, `@mosaicstack/memory`) need test coverage.
|
||||
- **Headless bootstrap** — admin password via env var is a credential on disk; needs clear documentation that `MOSAIC_ADMIN_PASSWORD` is intended for CI-only and should be rotated post-install.
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- `mosaicstack.dev/install.sh` vanity URL (blocked on marketing site work)
|
||||
- Uninstall for the `@mosaicstack/gateway` database contents — delegated to `mosaic gateway uninstall` semantics already in place
|
||||
- Signature/checksum verification of install scripts
|
||||
@@ -0,0 +1,41 @@
|
||||
# Tasks — Install UX Hardening
|
||||
|
||||
> Single-writer: orchestrator only. Workers read but never modify.
|
||||
>
|
||||
> **Mission:** install-ux-hardening-20260405
|
||||
> **Schema:** `| id | status | description | issue | agent | branch | depends_on | estimate | notes |`
|
||||
> **Status values:** `not-started` | `in-progress` | `done` | `blocked` | `failed` | `needs-qa`
|
||||
> **Agent values:** `codex` | `sonnet` | `haiku` | `opus` | `—` (auto)
|
||||
|
||||
## Milestone 1 — `mosaic uninstall` (IUH-M01)
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ------ | ------------------------------------------------------------------------------------------------------------------- | ----- | ------ | --------------------- | ---------- | -------- | ------------------------------------------------------ |
|
||||
| IUH-01-01 | done | Design install manifest schema (`~/.config/mosaic/.install-manifest.json`) — what install writes on first success | #425 | sonnet | feat/mosaic-uninstall | — | 8K | v1 schema in `install-manifest.ts` |
|
||||
| IUH-01-02 | done | `mosaic uninstall` TS command: `--framework`, `--cli`, `--gateway`, `--all`, `--keep-data`, `--yes`, `--dry-run` | #425 | sonnet | feat/mosaic-uninstall | IUH-01-01 | 25K | `uninstall.ts` |
|
||||
| IUH-01-03 | done | Reverse runtime asset linking in `~/.claude/` — restore `.mosaic-bak-*` if present, remove managed copies otherwise | #425 | sonnet | feat/mosaic-uninstall | IUH-01-02 | 12K | file list hardcoded from mosaic-link-runtime-assets |
|
||||
| IUH-01-04 | done | Reverse npmrc scope mapping and PATH edits made by `tools/install.sh` | #425 | sonnet | feat/mosaic-uninstall | IUH-01-02 | 8K | npmrc reversed; no PATH edits found in v0.0.24 install |
|
||||
| IUH-01-05 | done | Shell fallback: `tools/install.sh --uninstall` path for users without a working CLI | #425 | sonnet | feat/mosaic-uninstall | IUH-01-02 | 10K | |
|
||||
| IUH-01-06 | done | Vitest coverage: dry-run output, `--all`, `--keep-data`, partial state, missing manifest | #425 | sonnet | feat/mosaic-uninstall | IUH-01-05 | 15K | 14 new tests, 170 total |
|
||||
| IUH-01-07 | done | Code review (independent) + remediation | #425 | sonnet | feat/mosaic-uninstall | IUH-01-06 | 5K | |
|
||||
| IUH-01-08 | done | PR open, CI green, review, merge to `main`, close issue | #425 | sonnet | feat/mosaic-uninstall | IUH-01-07 | 3K | PR #429, merge 25cada77 |
|
||||
|
||||
## Milestone 2 — Wizard Remediation (IUH-M02)
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ------ | -------------------------------------------------------------------------------------------------------------- | ----- | ------ | ----------------------- | ---------- | -------- | ----------------------------------------------- |
|
||||
| IUH-02-01 | done | Password masking: replace plaintext `rl.question` in `bootstrapFirstUser` with masked TTY read + confirmation | #426 | sonnet | feat/wizard-remediation | IUH-01-08 | 8K | `prompter/masked-prompt.ts` |
|
||||
| IUH-02-02 | done | Hooks preview stage in wizard: show `framework/runtime/claude/hooks-config.json` entries + confirm prompt | #426 | sonnet | feat/wizard-remediation | IUH-02-01 | 12K | `stages/hooks-preview.ts`; finalize gating TODO |
|
||||
| IUH-02-03 | done | `mosaic config hooks list\|enable\|disable` subcommands | #426 | sonnet | feat/wizard-remediation | IUH-02-02 | 15K | `commands/config.ts` |
|
||||
| IUH-02-04 | done | Headless path: env-var driven `runConfigWizard` + `bootstrapFirstUser` (`MOSAIC_ASSUME_YES`, `MOSAIC_ADMIN_*`) | #426 | sonnet | feat/wizard-remediation | IUH-02-03 | 12K | |
|
||||
| IUH-02-05 | done | Tests + code review + PR merge | #426 | sonnet | feat/wizard-remediation | IUH-02-04 | 10K | PR #431, merge cd8b1f66 |
|
||||
|
||||
## Milestone 3 — Unified First-Run Wizard (IUH-M03)
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ------ | ----------------------------------------------------------------------------------------------------------- | ----- | ----- | ---------------------- | ---------- | -------- | ---------------------------------- |
|
||||
| IUH-03-01 | done | Design doc: unified state machine; decide whether `mosaic gateway install` becomes an internal wizard stage | #427 | opus | feat/unified-first-run | IUH-02-05 | 10K | scratchpad Session 5 |
|
||||
| IUH-03-02 | done | Refactor `runWizard` to invoke gateway install as a stage; drop the 10-minute session-file bridge | #427 | opus | feat/unified-first-run | IUH-03-01 | 25K | stages 11 & 12; bridge removed |
|
||||
| IUH-03-03 | done | Preserve backward-compat: `mosaic gateway install` still works as a standalone entry point | #427 | opus | feat/unified-first-run | IUH-03-02 | 10K | thin wrapper over stages |
|
||||
| IUH-03-04 | done | Tests + code review + PR merge | #427 | opus | feat/unified-first-run | IUH-03-03 | 12K | PR #433, merge 732f8a49; +15 tests |
|
||||
| IUH-03-05 | done | Bonus: honor `state.hooks.accepted` in finalize stage (closes M02 follow-up) | #427 | opus | feat/unified-first-run | IUH-03-04 | 5K | MOSAIC_SKIP_CLAUDE_HOOKS env flag |
|
||||
@@ -0,0 +1,74 @@
|
||||
# Mission Manifest — Install UX v2
|
||||
|
||||
> Persistent document tracking full mission scope, status, and session history.
|
||||
> Updated by the orchestrator at each phase transition and milestone completion.
|
||||
|
||||
## Mission
|
||||
|
||||
**ID:** install-ux-v2-20260405
|
||||
**Statement:** The install-ux-hardening mission shipped the plumbing (uninstall, masked password, hooks consent, unified flow, headless path), but the first real end-to-end run surfaced a critical regression and a collection of UX failings that make the wizard feel neither quick nor intelligent. This mission closes the bootstrap regression as a hotfix, then rethinks the first-run experience around a provider-first, intent-driven flow with a drill-down main menu and a genuinely fast quick-start.
|
||||
**Phase:** Closed
|
||||
**Current Milestone:** —
|
||||
**Progress:** 3 / 3 milestones
|
||||
**Status:** complete
|
||||
**Last Updated:** 2026-04-19 (archived during MVP manifest authoring; IUV-M03 substantively shipped via PR #446 — drill-down menu + provider-first flow + quick start; releases 0.0.27 → 0.0.29)
|
||||
**Archived to:** `docs/archive/missions/install-ux-v2-20260405/`
|
||||
**Parent Mission:** [install-ux-hardening-20260405](./archive/missions/install-ux-hardening-20260405/MISSION-MANIFEST.md) (complete — `mosaic-v0.0.25`)
|
||||
|
||||
## Context
|
||||
|
||||
Real-run testing of `@mosaicstack/[email protected]` uncovered:
|
||||
|
||||
1. **Critical:** admin bootstrap fails with HTTP 400 `property email should not exist` — `bootstrap.controller.ts` uses `import type { BootstrapSetupDto }`, erasing the class at runtime. Nest's `@Body()` falls back to plain `Object` metatype, and ValidationPipe with `forbidNonWhitelisted` rejects every property. One-character fix (drop the `type` keyword), but it blocks the happy path of the release that just shipped.
|
||||
2. The wizard reports `✔ Wizard complete` and `✔ Done` _after_ the bootstrap 400 — failure only propagates in headless mode (`wizard.ts:147`).
|
||||
3. The gateway port prompt does not prefill `14242` in the input buffer.
|
||||
4. `"What is Mosaic?"` intro copy does not mention Pi SDK (the actual agent runtime behind Claude/Codex/OpenCode).
|
||||
5. CORS origin prompt is confusing — the user should be able to supply an FQDN/hostname and have the system derive the CORS value.
|
||||
6. Skill / additional feature install section is unusable in practice.
|
||||
7. Quick-start asks far too many questions to be meaningfully "quick".
|
||||
8. No drill-down main menu — everything is a linear interrogation.
|
||||
9. Provider setup happens late and without intelligence. An OpenClaw-style provider-first flow would let the user describe what they want in natural language, have the agent expound on it, and have the agent choose its own name based on that intent.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [x] AC-1: Admin bootstrap completes successfully end-to-end on a fresh install (DTO value import, no forbidNonWhitelisted regression); covered by an integration or e2e test that exercises the real DTO binding. _(PR #440)_
|
||||
- [x] AC-2: Wizard fails loudly (non-zero exit, clear error) when the bootstrap stage returns `completed: false`, in both interactive and headless modes. No more silent `✔ Wizard complete` after a 400. _(PR #440)_
|
||||
- [x] AC-3: Gateway port prompt prefills `14242` in the input field (user can press Enter to accept). _(PR #440)_
|
||||
- [x] AC-4: `"What is Mosaic?"` intro copy mentions Pi SDK as the underlying agent runtime. _(PR #440)_
|
||||
- [x] AC-5: Release `mosaic-v0.0.26` tagged and published to the Gitea npm registry, unblocking the 0.0.25 happy path. _(tag: mosaic-v0.0.26, registry: 0.0.26 live)_
|
||||
- [ ] AC-6: CORS origin prompt replaced with FQDN/hostname input; CORS string is derived from that.
|
||||
- [ ] AC-7: Skill / additional feature install section is reworked until it is actually usable end-to-end (worker defines the concrete failure modes during diagnosis).
|
||||
- [ ] AC-8: First-run flow has a drill-down main menu with at least `Plugins` (Recommended / Custom), `Providers`, and the other top-level configuration groups. Linear interrogation is gone.
|
||||
- [ ] AC-9: `Quick Start` path completes with a minimal, curated set of questions (target: under 90 seconds for a returning user; define the exact baseline during design).
|
||||
- [ ] AC-10: Provider setup happens first, driven by a natural-language intake prompt. The agent expounds on the user's intent and chooses its own name based on that intent (OpenClaw-style). Naming is confirmable / overridable.
|
||||
- [ ] AC-11: All milestones ship as merged PRs with green CI and closed issues.
|
||||
|
||||
## Milestones
|
||||
|
||||
| # | ID | Name | Status | Branch | Issue | Started | Completed |
|
||||
| --- | ------- | ------------------------------------------------------------ | -------- | ---------------------- | ----- | ---------- | ---------- |
|
||||
| 1 | IUV-M01 | Hotfix: bootstrap DTO + wizard failure + port prefill + copy | complete | fix/bootstrap-hotfix | #436 | 2026-04-05 | 2026-04-05 |
|
||||
| 2 | IUV-M02 | UX polish: CORS/FQDN, skill installer rework | complete | feat/install-ux-polish | #437 | 2026-04-05 | 2026-04-05 |
|
||||
| 3 | IUV-M03 | Provider-first intelligent flow + drill-down main menu | complete | feat/install-ux-intent | #438 | 2026-04-05 | 2026-04-19 |
|
||||
|
||||
## Subagent Delegation Plan
|
||||
|
||||
| Milestone | Recommended Tier | Rationale |
|
||||
| --------- | ---------------- | --------------------------------------------------------------------- |
|
||||
| IUV-M01 | sonnet | Tight bug cluster with known fix sites + small release cycle |
|
||||
| IUV-M02 | sonnet | UX rework, moderate surface, diagnostic-heavy for the skill installer |
|
||||
| IUV-M03 | opus | Architectural redesign of first-run flow, state machine + LLM intake |
|
||||
|
||||
## Risks
|
||||
|
||||
- **Hotfix regression surface** — the `import type` → `import` fix on the DTO class is one character but needs an integration test that binds the real DTO, not just a controller unit test, to prevent the same class-erasure regression from sneaking back in.
|
||||
- **LLM-driven intake latency / offline** — M03's provider-first intent flow assumes an available LLM call to expound on user input and choose a name. Offline installs need a deterministic fallback.
|
||||
- **Menu vs. linear back-compat** — M03 changes the top-level flow shape; existing `tools/install.sh --yes` + env-var headless path must continue to work.
|
||||
- **Scope creep in M03** — "redesign the wizard" can absorb arbitrary work. Keep it bounded with explicit non-goals.
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Migrating the wizard to a GUI / web UI (still terminal-first)
|
||||
- Replacing the Gitea registry or the Woodpecker publish pipeline
|
||||
- Multi-tenant / multi-user onboarding (still single-admin bootstrap)
|
||||
- Reworking `mosaic uninstall` (M01 of the parent mission — stable)
|
||||
@@ -0,0 +1,39 @@
|
||||
# Tasks — Install UX v2
|
||||
|
||||
> Single-writer: orchestrator only. Workers read but never modify.
|
||||
>
|
||||
> **Mission:** install-ux-v2-20260405
|
||||
> **Schema:** `| id | status | description | issue | agent | branch | depends_on | estimate | notes |`
|
||||
> **Status values:** `not-started` | `in-progress` | `done` | `blocked` | `failed` | `needs-qa`
|
||||
> **Agent values:** `codex` | `sonnet` | `haiku` | `opus` | `—` (auto)
|
||||
|
||||
## Milestone 1 — Hotfix: bootstrap DTO + wizard failure + port prefill + copy (IUV-M01)
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | ------ | -------------------- | ---------- | -------- | --------------------------------------------------------------------------------------- |
|
||||
| IUV-01-01 | done | Fix `apps/gateway/src/admin/bootstrap.controller.ts:16` — switch `import type { BootstrapSetupDto }` to a value import so Nest's `@Body()` binds the real class | #436 | sonnet | fix/bootstrap-hotfix | — | 3K | PR #440 merged `0ae932ab` |
|
||||
| IUV-01-02 | done | Add integration / e2e test that POSTs `/api/bootstrap/setup` with `{name,email,password}` against a real Nest app instance and asserts 201 — NOT a mocked controller unit test | #436 | sonnet | fix/bootstrap-hotfix | IUV-01-01 | 10K | `apps/gateway/src/admin/bootstrap.e2e.spec.ts` — 4 tests; unplugin-swc added for vitest |
|
||||
| IUV-01-03 | done | `packages/mosaic/src/wizard.ts:147` — propagate `!bootstrapResult.completed` as a wizard failure in **interactive** mode too (not only headless); non-zero exit + no `✔ Wizard complete` line | #436 | sonnet | fix/bootstrap-hotfix | IUV-01-02 | 5K | removed `&& headlessRun` guard |
|
||||
| IUV-01-04 | done | Gateway port prompt prefills `14242` in the input buffer — investigate why `promptPort`'s `defaultValue` isn't reaching the user-visible input | #436 | sonnet | fix/bootstrap-hotfix | IUV-01-03 | 5K | added `initialValue` through prompter interface → clack |
|
||||
| IUV-01-05 | done | `"What is Mosaic?"` intro copy updated to mention Pi SDK as the underlying agent runtime (alongside Claude Code / Codex / OpenCode) | #436 | sonnet | fix/bootstrap-hotfix | IUV-01-04 | 2K | `packages/mosaic/src/stages/welcome.ts` |
|
||||
| IUV-01-06 | done | Tests + code review + PR merge + tag `mosaic-v0.0.26` + Gitea release + npm registry republish | #436 | sonnet | fix/bootstrap-hotfix | IUV-01-05 | 10K | PRs #440/#441/#442 merged; tag `mosaic-v0.0.26`; registry latest=0.0.26 ✓ |
|
||||
|
||||
## Milestone 2 — UX polish: CORS/FQDN, skill installer rework (IUV-M02)
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ---------------------- | ---------- | -------- | ---------------------------------------------------------------------- |
|
||||
| IUV-02-01 | done | Replace CORS origin prompt with FQDN / hostname input; derive the CORS value internally; default to `localhost` with clear help text | #437 | sonnet | feat/install-ux-polish | — | 10K | `deriveCorsOrigin()` pure fn; MOSAIC_HOSTNAME headless var; PR #444 |
|
||||
| IUV-02-02 | done | Diagnose and document the concrete failure modes of the current skill / additional feature install section end-to-end | #437 | sonnet | feat/install-ux-polish | IUV-02-01 | 8K | selection→install gap, silent catch{}, no whitelist concept |
|
||||
| IUV-02-03 | done | Rework the skill installer so it is usable end-to-end (selection, install, verify, failure reporting) | #437 | sonnet | feat/install-ux-polish | IUV-02-02 | 20K | MOSAIC_INSTALL_SKILLS env var whitelist; SyncSkillsResult typed return |
|
||||
| IUV-02-04 | done | Tests + code review + PR merge | #437 | sonnet | feat/install-ux-polish | IUV-02-03 | 10K | 18 new tests (13 CORS + 5 skills); PR #444 merged `172bacb3` |
|
||||
|
||||
## Milestone 3 — Provider-first intelligent flow + drill-down main menu (IUV-M03)
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ----- | ---------------------- | ---------- | -------- | ------------------------------------------------------------- |
|
||||
| IUV-03-01 | not-started | Design doc: new first-run state machine — main menu (Plugins / Providers / …), Quick Start vs Custom paths, provider-first flow, intent intake + naming loop | #438 | opus | feat/install-ux-intent | — | 15K | scratchpad + explicit non-goals |
|
||||
| IUV-03-02 | not-started | Implement drill-down main menu (Plugins: Recommended / Custom, Providers, …) as the top-level entry point of `mosaic wizard` | #438 | opus | feat/install-ux-intent | IUV-03-01 | 25K | |
|
||||
| IUV-03-03 | not-started | Quick Start path: curated minimum question set — define the exact baseline, delete everything else from the fast path | #438 | opus | feat/install-ux-intent | IUV-03-02 | 15K | |
|
||||
| IUV-03-04 | not-started | Provider-first natural-language intake: user describes intent → agent expounds → agent proposes a name (confirmable / overridable) — OpenClaw-style | #438 | opus | feat/install-ux-intent | IUV-03-03 | 25K | offline fallback required (deterministic default name + path) |
|
||||
| IUV-03-05 | not-started | Preserve backward-compat: headless path (`MOSAIC_ASSUME_YES=1` + env vars) still works end-to-end; `tools/install.sh --yes` unchanged | #438 | opus | feat/install-ux-intent | IUV-03-04 | 10K | |
|
||||
| IUV-03-06 | not-started | Tests + code review + PR merge + `mosaic-v0.0.27` release | #438 | opus | feat/install-ux-intent | IUV-03-05 | 15K | |
|
||||
@@ -0,0 +1,227 @@
|
||||
# IUV-M03 Design: Provider-first intelligent flow + drill-down main menu
|
||||
|
||||
**Issue:** #438
|
||||
**Branch:** `feat/install-ux-intent`
|
||||
**Date:** 2026-04-05
|
||||
|
||||
## 1. New first-run state machine
|
||||
|
||||
The linear 12-stage interrogation is replaced with a menu-driven architecture.
|
||||
|
||||
### Flow overview
|
||||
|
||||
```
|
||||
Welcome banner
|
||||
|
|
||||
v
|
||||
Detect existing install (auto)
|
||||
|
|
||||
v
|
||||
Main Menu (loop)
|
||||
|-- Quick Start -> provider key + admin creds -> finalize
|
||||
|-- Providers -> LLM API key config
|
||||
|-- Agent Identity -> intent intake + naming (deterministic)
|
||||
|-- Skills -> recommended / custom selection
|
||||
|-- Gateway -> port, storage tier, hostname, CORS
|
||||
|-- Advanced -> SOUL.md, USER.md, TOOLS.md, runtimes, hooks
|
||||
|-- Finish & Apply -> finalize + gateway bootstrap
|
||||
v
|
||||
Done
|
||||
```
|
||||
|
||||
### Menu navigation
|
||||
|
||||
- Main menu is a `select` prompt. Each option drills into a sub-flow.
|
||||
- Completing a section returns to the main menu.
|
||||
- Menu items show completion state: `[done]` hint after configuration.
|
||||
- `Finish & Apply` is always last and requires at minimum a provider key (or explicit skip).
|
||||
- The menu tracks configured sections in `WizardState.completedSections`.
|
||||
|
||||
### Headless bypass
|
||||
|
||||
When `MOSAIC_ASSUME_YES=1` or `!process.stdin.isTTY`, the entire menu is skipped.
|
||||
The wizard runs: defaults + env var overrides -> finalize -> gateway config -> bootstrap.
|
||||
This preserves full backward compatibility with `tools/install.sh --yes`.
|
||||
|
||||
## 2. Quick Start path
|
||||
|
||||
Target: 3-5 questions max. Under 90 seconds for a returning user.
|
||||
|
||||
### Questions asked
|
||||
|
||||
1. **Provider API key** (Anthropic/OpenAI) - `text` prompt with paste support
|
||||
2. **Admin email** - `text` prompt
|
||||
3. **Admin password** - masked + confirmed
|
||||
|
||||
### Questions skipped (with defaults)
|
||||
|
||||
| Setting | Default | Rationale |
|
||||
| ---------------------------- | ------------------------------- | ---------------------- |
|
||||
| Agent name | "Mosaic" | Generic but branded |
|
||||
| Port | 14242 | Standard default |
|
||||
| Storage tier | local | No external deps |
|
||||
| Hostname | localhost | Dev-first |
|
||||
| CORS origin | http://localhost:3000 | Standard web UI port |
|
||||
| Skills | recommended set | Curated by maintainers |
|
||||
| Runtimes | auto-detected | No user input needed |
|
||||
| Communication style | direct | Most popular choice |
|
||||
| SOUL.md / USER.md / TOOLS.md | template defaults | Can customize later |
|
||||
| Hooks | auto-install if Claude detected | Safe default |
|
||||
|
||||
### Flow
|
||||
|
||||
```
|
||||
Quick Start selected
|
||||
-> "Paste your LLM API key (Anthropic recommended):"
|
||||
-> [auto-detect provider from key prefix: sk-ant-* = Anthropic, sk-* = OpenAI]
|
||||
-> Apply all defaults
|
||||
-> Run finalize (sync framework, write configs, link assets, sync skills)
|
||||
-> Run gateway config (headless-style with defaults + provided key)
|
||||
-> "Admin email:"
|
||||
-> "Admin password:" (masked + confirm)
|
||||
-> Run gateway bootstrap
|
||||
-> Done
|
||||
```
|
||||
|
||||
## 3. Provider-first flow
|
||||
|
||||
Provider configuration (currently buried in gateway-config stage as "ANTHROPIC_API_KEY")
|
||||
moves to a dedicated top-level menu item and is the first question in Quick Start.
|
||||
|
||||
### Provider detection
|
||||
|
||||
The API key prefix determines the provider:
|
||||
|
||||
- `sk-ant-api03-*` -> Anthropic (Claude)
|
||||
- `sk-*` -> OpenAI
|
||||
- Empty/skipped -> no provider (gateway starts without LLM access)
|
||||
|
||||
### Storage
|
||||
|
||||
The provider key is stored in the gateway `.env` as `ANTHROPIC_API_KEY` or `OPENAI_API_KEY`.
|
||||
For Quick Start, this replaces the old interactive prompt in `collectAndWriteConfig`.
|
||||
|
||||
### Menu section: "Providers"
|
||||
|
||||
In the drill-down menu, "Providers" lets users:
|
||||
|
||||
1. Enter/change their API key
|
||||
2. See which provider was detected
|
||||
3. Optionally configure a second provider
|
||||
|
||||
For v0.0.27, we support Anthropic and OpenAI keys only. The key is stored
|
||||
in `WizardState` and written during finalize.
|
||||
|
||||
## 4. Intent intake + naming (deterministic fallback - Option B)
|
||||
|
||||
### Rationale
|
||||
|
||||
At install time, the LLM provider may not be configured yet (chicken-and-egg).
|
||||
We use **Option B: deterministic advisor** for the install wizard.
|
||||
|
||||
### Flow (Agent Identity menu section)
|
||||
|
||||
```
|
||||
1. "What will this agent primarily help you with?"
|
||||
-> Select from presets:
|
||||
- General purpose assistant
|
||||
- Software development
|
||||
- DevOps & infrastructure
|
||||
- Research & analysis
|
||||
- Content & writing
|
||||
- Custom (free text description)
|
||||
|
||||
2. System proposes a thematic name based on selection:
|
||||
- General purpose -> "Mosaic"
|
||||
- Software development -> "Forge"
|
||||
- DevOps & infrastructure -> "Sentinel"
|
||||
- Research & analysis -> "Atlas"
|
||||
- Content & writing -> "Muse"
|
||||
- Custom -> "Mosaic" (default)
|
||||
|
||||
3. "Your agent will be named 'Forge'. Press Enter to accept or type a new name:"
|
||||
-> User confirms or overrides
|
||||
```
|
||||
|
||||
### Storage
|
||||
|
||||
- Agent name -> `WizardState.soul.agentName` -> written to SOUL.md
|
||||
- Intent category -> `WizardState.agentIntent` (new field) -> written to `~/.config/mosaic/agent.json`
|
||||
|
||||
### Post-install LLM-powered intake (future)
|
||||
|
||||
A future `mosaic configure identity` command can use the configured LLM to:
|
||||
|
||||
- Accept free-text intent description
|
||||
- Generate an expounded persona
|
||||
- Propose a contextual name
|
||||
|
||||
This is explicitly out of scope for the install wizard.
|
||||
|
||||
## 5. Headless backward-compat
|
||||
|
||||
### Supported env vars (unchanged)
|
||||
|
||||
| Variable | Used by |
|
||||
| -------------------------- | ---------------------------------------------- |
|
||||
| `MOSAIC_ASSUME_YES=1` | Skip all prompts, use defaults + env overrides |
|
||||
| `MOSAIC_ADMIN_NAME` | Gateway bootstrap |
|
||||
| `MOSAIC_ADMIN_EMAIL` | Gateway bootstrap |
|
||||
| `MOSAIC_ADMIN_PASSWORD` | Gateway bootstrap |
|
||||
| `MOSAIC_GATEWAY_PORT` | Gateway config |
|
||||
| `MOSAIC_HOSTNAME` | Gateway config (CORS derivation) |
|
||||
| `MOSAIC_CORS_ORIGIN` | Gateway config (full override) |
|
||||
| `MOSAIC_STORAGE_TIER` | Gateway config (local/team) |
|
||||
| `MOSAIC_DATABASE_URL` | Gateway config (team tier) |
|
||||
| `MOSAIC_VALKEY_URL` | Gateway config (team tier) |
|
||||
| `MOSAIC_ANTHROPIC_API_KEY` | Provider config |
|
||||
|
||||
### New env vars
|
||||
|
||||
| Variable | Purpose |
|
||||
| --------------------- | ----------------------------------------- |
|
||||
| `MOSAIC_AGENT_NAME` | Override agent name in headless mode |
|
||||
| `MOSAIC_AGENT_INTENT` | Override intent category in headless mode |
|
||||
|
||||
### `tools/install.sh --yes`
|
||||
|
||||
The install script sets `MOSAIC_ASSUME_YES=1` and passes through env vars.
|
||||
No changes needed to the script itself. The new wizard detects headless mode
|
||||
at the top of `runWizard` and runs a linear path identical to the old flow.
|
||||
|
||||
## 6. Explicit non-goals
|
||||
|
||||
- **No GUI** — this is a terminal wizard only
|
||||
- **No multi-user install** — single-user, single-machine
|
||||
- **No registry changes** — npm publish flow is unchanged
|
||||
- **No LLM calls during install** — deterministic fallback only
|
||||
- **No new dependencies** — uses existing @clack/prompts and picocolors
|
||||
- **No changes to gateway API** — only the wizard orchestration changes
|
||||
- **No changes to tools/install.sh** — headless compat maintained via env vars
|
||||
|
||||
## 7. Implementation plan
|
||||
|
||||
### Files to modify
|
||||
|
||||
1. `packages/mosaic/src/types.ts` — add `MenuSection`, `AgentIntent`, `completedSections`, `agentIntent`, `providerKey`, `providerType` to WizardState
|
||||
2. `packages/mosaic/src/wizard.ts` — replace linear flow with menu loop
|
||||
3. `packages/mosaic/src/stages/mode-select.ts` — becomes the main menu
|
||||
4. `packages/mosaic/src/stages/provider-setup.ts` — new: provider key collection
|
||||
5. `packages/mosaic/src/stages/agent-intent.ts` — new: intent intake + naming
|
||||
6. `packages/mosaic/src/stages/menu-gateway.ts` — new: gateway sub-menu wrapper
|
||||
7. `packages/mosaic/src/stages/quick-start.ts` — new: quick start linear path
|
||||
8. `packages/mosaic/src/constants.ts` — add intent presets and name mappings
|
||||
9. `packages/mosaic/package.json` — version bump 0.0.26 -> 0.0.27
|
||||
|
||||
### Files to add (tests)
|
||||
|
||||
1. `packages/mosaic/src/stages/wizard-menu.spec.ts` — menu navigation tests
|
||||
2. `packages/mosaic/src/stages/quick-start.spec.ts` — quick start path tests
|
||||
3. `packages/mosaic/src/stages/agent-intent.spec.ts` — intent + naming tests
|
||||
4. `packages/mosaic/src/stages/provider-setup.spec.ts` — provider detection tests
|
||||
|
||||
### Migration strategy
|
||||
|
||||
The existing stage functions remain intact. The menu system wraps them —
|
||||
each menu item calls the appropriate stage function(s). The linear headless
|
||||
path calls them in the same order as before.
|
||||
@@ -0,0 +1,173 @@
|
||||
# Install UX v2 — Orchestrator Scratchpad
|
||||
|
||||
## Session 1 — 2026-04-05 (orchestrator scaffold)
|
||||
|
||||
### Trigger
|
||||
|
||||
Real-run testing of `@mosaicstack/[email protected]` (fresh install of the release we just shipped from the parent mission `install-ux-hardening-20260405`) surfaced a critical regression and a cluster of UX failings. User feedback verbatim:
|
||||
|
||||
> The skill/additional feature installation section of install.sh is unsable
|
||||
> The "quick-start" is asking way too many questions. This process should be much faster to get a quick start.
|
||||
> The installater should have a main menu that allows for a drill-down install approach.
|
||||
> "Plugins" — Install Recommended Plugins / Custom
|
||||
> "Providers" — …
|
||||
> The gateway port is not prefilling with 14242 for default
|
||||
> What is the CORS origin for? Is that the webUI that isn't working yet? Maybe we should ask for the fqdn/hostname instead? There must be a better way to handle this.
|
||||
|
||||
Plus the critical bug, reproduced verbatim:
|
||||
|
||||
```
|
||||
◇ Admin email
|
||||
│ [email protected]
|
||||
Admin password (min 8 chars): ****************
|
||||
Confirm password: ****************
|
||||
│
|
||||
▲ Bootstrap failed (400): {"message":["property email should not exist","property password should not exist"],"error":"Bad Request","statusCode":400}
|
||||
✔ Wizard complete.
|
||||
✔ Install manifest written: /home/jarvis/.config/mosaic/.install-manifest.json
|
||||
|
||||
✔ Done.
|
||||
```
|
||||
|
||||
Note the `✔ Wizard complete` and `✔ Done` lines **after** the 400. That's a second bug — failure didn't propagate in interactive mode.
|
||||
|
||||
### Diagnosis — orchestrator pre-scope
|
||||
|
||||
To avoid handing workers a vague prompt, pre-identified the concrete fix sites:
|
||||
|
||||
**Bug 1 (critical) — DTO class erasure.** `apps/gateway/src/admin/bootstrap.controller.ts:16`:
|
||||
|
||||
```ts
|
||||
import type { BootstrapSetupDto, BootstrapStatusDto, BootstrapResultDto } from './bootstrap.dto.js';
|
||||
```
|
||||
|
||||
`import type` erases the class at runtime. `@Body() dto: BootstrapSetupDto` then has no runtime metatype — `design:paramtypes` reflects `Object`. Nest's `ValidationPipe` with `whitelist: true` + `forbidNonWhitelisted: true` receives a plain Object metatype, treats every incoming property as non-whitelisted, and 400s with `"property email should not exist", "property password should not exist"`.
|
||||
|
||||
**One-character fix:** drop the `type` keyword on the `BootstrapSetupDto` import. `BootstrapStatusDto` and `BootstrapResultDto` are fine as type-only imports because they're used only in return type positions, not as `@Body()` metatypes.
|
||||
|
||||
Must be covered by an **integration test that binds through Nest**, not a controller unit test that imports the DTO directly — the unit test path would pass even with `import type` because it constructs the pipe manually. An e2e test with `@nestjs/testing` + `supertest` against the real `/api/bootstrap/setup` endpoint is the right guard.
|
||||
|
||||
**Bug 2 — interactive silent failure.** `packages/mosaic/src/wizard.ts:147-150`:
|
||||
|
||||
```ts
|
||||
if (!bootstrapResult.completed && headlessRun) {
|
||||
prompter.warn('Admin bootstrap failed in headless mode — aborting wizard.');
|
||||
process.exit(1);
|
||||
}
|
||||
```
|
||||
|
||||
The guard is `&& headlessRun`. In interactive mode, `completed: false` is silently swallowed and the wizard continues to the success lines. Fix: propagate failure in both modes. Decision for the worker — either `throw` or `process.exit(1)` with a clear error.
|
||||
|
||||
**Bug 3 — port prefill.** `packages/mosaic/src/stages/gateway-config.ts:77-88`:
|
||||
|
||||
```ts
|
||||
const raw = await p.text({
|
||||
message: 'Gateway port',
|
||||
defaultValue: defaultPort.toString(),
|
||||
...
|
||||
});
|
||||
```
|
||||
|
||||
The stage is passing `defaultValue`. Either the `WizardPrompter.text` adapter is dropping it, or the underlying `@clack/prompts` call expects `initialValue` (which actually prefills the buffer) vs `defaultValue` (which is used only if the user submits an empty string). Worker should verify the adapter and likely switch to `initialValue` semantics so the user sees `14242` in the field.
|
||||
|
||||
**Bug 4 — Pi SDK copy gap.** The `"What is Mosaic?"` intro text enumerates Claude Code, Codex, and OpenCode but never mentions Pi SDK, which is the actual agent runtime behind those frontends. Purely a copy edit — find the string, add Pi SDK.
|
||||
|
||||
### Mission shape
|
||||
|
||||
Three milestones, three tracks, different tiers:
|
||||
|
||||
1. **IUV-M01 Hotfix** (sonnet) — the four bugs above + release `mosaic-v0.0.26`. Small, fast, unblocks the 0.0.25 happy path.
|
||||
2. **IUV-M02 UX polish** (sonnet) — CORS origin → FQDN/hostname abstraction; diagnose and rework the skill installer section. Diagnostic-heavy.
|
||||
3. **IUV-M03 Provider-first intelligent flow** (opus) — the big one: drill-down main menu, Quick Start path that's actually quick, provider-first natural-language intake with agent self-naming (OpenClaw-style). Architectural.
|
||||
|
||||
Sequencing: strict. M01 ships first as a hotfix release (mosaic-v0.0.26). M02 is diagnostic-heavy and can share groundwork with M03 but ships separately for clean release notes. M03 is the architectural anchor and lands last as `mosaic-v0.0.27`.
|
||||
|
||||
### Open design questions (to be resolved by workers, not pre-decided)
|
||||
|
||||
- M01: does `process.exit(1)` vs `throw` matter for how `tools/install.sh` surfaces the error? Worker should check the install.sh call site and pick the behavior that surfaces cleanly.
|
||||
- M03: what LLM call powers the intent intake, and what's the offline fallback? Options: (a) reuse the provider the user is configuring (chicken-and-egg — provider setup hasn't happened yet), (b) a bundled deterministic "advisor" that hard-codes common intents, (c) require a provider key up-front before intake. Design doc (IUV-03-01) must resolve.
|
||||
- M03: is the "agent self-naming" persistent across all future `mosaic` invocations, or a per-session nickname? Probably persistent — lives in `~/.config/mosaic/agent.json` or similar. Worker to decide + document.
|
||||
|
||||
### Non-goals for this mission
|
||||
|
||||
- No GUI / web UI
|
||||
- No registry / pipeline migration
|
||||
- No multi-user / multi-tenant onboarding
|
||||
- No rework of `mosaic uninstall` (stable from parent mission)
|
||||
|
||||
### Known tooling caveats (carry forward from parent mission)
|
||||
|
||||
- `issue-create.sh` / `pr-create.sh` wrappers have an `eval` bug with multiline bodies — use Gitea REST API fallback with `load_credentials gitea-mosaicstack`
|
||||
- `pr-ci-wait.sh` reports `state=unknown` against Woodpecker (combined-status endpoint gap) — use `tea pr` glyphs or poll the commit status endpoint directly
|
||||
- Protected `main`, squash-merge only, PR-required
|
||||
- CI queue guard before push/merge: `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge`
|
||||
|
||||
### Next action
|
||||
|
||||
1. Create Gitea issues for M01, M02, M03
|
||||
2. Open the mission-scaffold docs PR (same pattern as parent mission's PR #430)
|
||||
3. After merge, delegate IUV-M01 to a sonnet subagent in an isolated worktree with the concrete fix-site pointers above
|
||||
|
||||
## Session 2 — 2026-04-05 (IUV-M01 delivery + close-out)
|
||||
|
||||
### Outcome
|
||||
|
||||
IUV-M01 shipped. `mosaic-v0.0.26` released and registry latest confirmed `0.0.26`.
|
||||
|
||||
### PRs merged
|
||||
|
||||
| PR | Title | Merge |
|
||||
| ---- | ------------------------------------------------------------------------ | -------- |
|
||||
| #440 | fix: bootstrap hotfix — DTO erasure, wizard failure, port prefill, copy | 0ae932ab |
|
||||
| #441 | fix: add vitest.config.ts to eslint allowDefaultProject (#440 build fix) | c08aa6fa |
|
||||
| #442 | docs: mark IUV-M01 complete — mosaic-v0.0.26 released | 78388437 |
|
||||
|
||||
### Bugs fixed (all 4 in worker's PR #440)
|
||||
|
||||
1. **DTO class erasure** — `apps/gateway/src/admin/bootstrap.controller.ts:16` — dropped `type` from `import { BootstrapSetupDto }`. Guarded by new e2e test `bootstrap.e2e.spec.ts` (4 cases) that binds through a real Nest app with `ValidationPipe { whitelist, forbidNonWhitelisted }`. Test suite needed `unplugin-swc` in `apps/gateway/vitest.config.ts` to emit `decoratorMetadata` (tsx/esbuild can't).
|
||||
2. **Wizard silent failure** — `packages/mosaic/src/wizard.ts` — removed the `&& headlessRun` guard so `!bootstrapResult.completed` now aborts in both modes.
|
||||
3. **Port prefill** — root cause was clack's `defaultValue` vs `initialValue` semantics (`defaultValue` only fills on empty submit, `initialValue` prefills the buffer). Added an `initialValue` field to `WizardPrompter.text()` interface, threaded through clack and headless prompters, switched `gateway-config.ts` port/url prompts to use it.
|
||||
4. **Pi SDK copy** — `packages/mosaic/src/stages/welcome.ts` — intro copy now lists Pi SDK.
|
||||
|
||||
### Mid-delivery hiccup — tsconfig/eslint cross-contamination
|
||||
|
||||
Worker's initial approach added `vitest.config.ts` to `apps/gateway/tsconfig.json`'s `include` to appease the eslint parser. That broke `pnpm --filter @mosaicstack/gateway build` with TS6059 (`vitest.config.ts` outside `rootDir: "src"`). The publish pipeline on the `#440` merge commit failed.
|
||||
|
||||
**Correct fix** (worker's PR #441): leave `tsconfig.json` clean (`include: ["src/**/*"]`) and instead add the file to `allowDefaultProject` in the root `eslint.config.mjs`. This keeps the tsc program strict while letting eslint resolve a parser project for the standalone config file.
|
||||
|
||||
**Pattern to remember**: when adding root-level `.ts` config files (vitest, build scripts) to a package with `rootDir: "src"`, the eslint parser project conflict is solved with `allowDefaultProject`, NEVER by widening tsconfig include. I had independently arrived at the same fix on a branch before the worker shipped #441 — deleted the duplicate.
|
||||
|
||||
### Residual follow-ups carried forward
|
||||
|
||||
1. Headless prompter fallback order: worker set `initialValue > defaultValue` in the headless path. Correct semantic, but any future headless test that explicitly depends on `defaultValue` precedence will need review.
|
||||
2. Vitest + SWC decorator metadata pattern is now the blessed approach for NestJS e2e tests in this monorepo. Any other package that adds NestJS e2e tests should mirror `apps/gateway/vitest.config.ts`.
|
||||
|
||||
### Next action
|
||||
|
||||
- Close out orchestrator doc sync (this commit): mark M01 subtasks done in `TASKS.md`, update manifest phase to Execution, commit scratchpad session 2, PR to main.
|
||||
- After merge, delegate IUV-M02 (sonnet, isolated worktree). Dependencies: IUV-02-01 (CORS→FQDN) starts unblocked since M01 is released; first real task for the M02 worker is diagnosing the skill installer failure modes (IUV-02-02) against the fresh 0.0.26 install.
|
||||
|
||||
## Session 3 — 2026-04-05 (IUV-M02 delivery + close-out)
|
||||
|
||||
### Outcome
|
||||
|
||||
IUV-M02 shipped. PR #444 merged (`172bacb3`), issue #437 closed. 18 new tests (13 CORS derivation, 5 skill sync).
|
||||
|
||||
### Changes
|
||||
|
||||
**CORS → FQDN (IUV-02-01):**
|
||||
|
||||
- `packages/mosaic/src/stages/gateway-config.ts` — replaced raw "CORS origin" text prompt with "Web UI hostname" (default: `localhost`). Added HTTPS follow-up for remote hosts. Pure `deriveCorsOrigin(hostname, port, useHttps?)` function exported for testability.
|
||||
- Headless: `MOSAIC_HOSTNAME` env var as friendly alternative; `MOSAIC_CORS_ORIGIN` still works as full override.
|
||||
- `packages/mosaic/src/types.ts` — added `hostname?: string` to `GatewayState`.
|
||||
|
||||
**Skill installer rework (IUV-02-02 + IUV-02-03):**
|
||||
|
||||
- Root cause confirmed: `syncSkills()` in `finalize.ts` ignored `state.selectedSkills` entirely. The multiselect UI was a no-op.
|
||||
- `packages/mosaic/src/stages/finalize.ts` — `syncSkills()` rewritten to accept `selectedSkills[]`, returns typed `SyncSkillsResult`, passes `MOSAIC_INSTALL_SKILLS` (colon-separated) as env var to the bash script.
|
||||
- `packages/mosaic/framework/tools/_scripts/mosaic-sync-skills` — added bash associative array whitelist filter keyed on `MOSAIC_INSTALL_SKILLS`. When set, only whitelisted skills are linked. Empty/unset = all skills (legacy behavior preserved for `mosaic sync` outside wizard).
|
||||
- Failure surfaces: silent `catch {}` replaced with typed error reporting through `p.warn()`.
|
||||
|
||||
### Next action
|
||||
|
||||
- Delegate IUV-M03 (opus, isolated worktree) — the architectural milestone: provider-first intelligent flow, drill-down main menu, Quick Start fast path, agent self-naming. This is the biggest piece of the mission.
|
||||
@@ -0,0 +1,30 @@
|
||||
# Tasks — Storage Abstraction Retrofit
|
||||
|
||||
> Single-writer: orchestrator only. Workers read but never modify.
|
||||
>
|
||||
> **Mission:** Decouple gateway from hardcoded Postgres/Valkey backends. Introduce interface-driven middleware so the gateway is backend-agnostic. Default to local tier (SQLite + JSON) for zero-dependency installs.
|
||||
>
|
||||
> **`agent` column values:** `codex` | `sonnet` | `haiku` | `glm-5` | `opus` | `—` (auto/default)
|
||||
|
||||
| id | status | agent | description | tokens |
|
||||
| --------- | ----------- | ------ | ---------------------------------------------------------------- | ------ |
|
||||
| SA-P1-001 | done | sonnet | Define QueueAdapter interface in packages/queue/src/types.ts | 3K |
|
||||
| SA-P1-002 | done | sonnet | Define StorageAdapter interface in packages/storage/src/types.ts | 3K |
|
||||
| SA-P1-003 | done | sonnet | Define MemoryAdapter interface in packages/memory/src/types.ts | 3K |
|
||||
| SA-P1-004 | done | sonnet | Create adapter factory pattern + config types | 3K |
|
||||
| SA-P2-001 | done | sonnet | Refactor @mosaicstack/queue: wrap ioredis as BullMQ adapter | 3K |
|
||||
| SA-P2-002 | done | sonnet | Create @mosaicstack/storage: wrap Drizzle as Postgres adapter | 6K |
|
||||
| SA-P2-003 | done | sonnet | Refactor @mosaicstack/memory: extract pgvector adapter | 4K |
|
||||
| SA-P2-004 | done | sonnet | Update gateway modules to use factories + DI tokens | 5K |
|
||||
| SA-P2-005 | done | opus | Verify Phase 2: all tests pass, typecheck clean | — |
|
||||
| SA-P3-001 | done | sonnet | Implement local queue adapter: JSON file persistence | 5K |
|
||||
| SA-P3-002 | done | sonnet | Implement SQLite storage adapter with better-sqlite3 | 8K |
|
||||
| SA-P3-003 | done | sonnet | Implement keyword memory adapter — no vector dependency | 4K |
|
||||
| SA-P3-004 | done | opus | Verify Phase 3: 42 new tests, 347 total passing | — |
|
||||
| SA-P4-001 | done | sonnet | MosaicConfig schema + loader with tier auto-detection | 6K |
|
||||
| SA-P4-002 | done | sonnet | CLI: mosaic gateway init — interactive wizard | 4K |
|
||||
| SA-P4-003 | done | sonnet | CLI: mosaic gateway start/stop/status lifecycle | 5K |
|
||||
| SA-P4-004 | done | opus | Verify Phase 4: 381 tests passing, 40/40 tasks clean | — |
|
||||
| SA-P5-001 | not-started | codex | Migration tooling: mosaic storage export/import | — |
|
||||
| SA-P5-002 | not-started | codex | Docker Compose profiles: local vs team | — |
|
||||
| SA-P5-003 | not-started | codex | Final verification + docs: README, architecture diagram | — |
|
||||
@@ -0,0 +1,139 @@
|
||||
# Mosaic Framework Consistency Audit
|
||||
|
||||
Date: 2026-02-17
|
||||
Scope:
|
||||
|
||||
- `~/src/mosaic-bootstrap`
|
||||
- `~/src/agent-skills`
|
||||
- Installed runtime check: `~/.config/mosaic`
|
||||
|
||||
## Findings
|
||||
|
||||
| ID | Severity | Finding | Status |
|
||||
| ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------ | ---------- |
|
||||
| MF-001 | High | QA rails scripts referenced invalid paths (missing `/qa/` segment) for `qa-hook-handler.sh` calls. | Remediated |
|
||||
| MF-002 | Medium | Conditional-loading detection was biased to legacy `agent-guides` markers and missed Mosaic `guides`/`Conditional Context` phrasing. | Remediated |
|
||||
| MF-003 | Medium | Framework spec text still described detection as `agent-guides`-based only. | Remediated |
|
||||
| MF-004 | Medium | Runtime Claude overlay hardcoded skill count, which drifts over time. | Remediated |
|
||||
| MF-005 | High | `pr-reviewer` skill used legacy `~/.claude/scripts/git` and `~/.claude/skills` paths. | Remediated |
|
||||
| MF-006 | Medium | `using-git-worktrees` prioritized `CLAUDE.md` only and ignored Mosaic-first docs (`AGENTS.md`/`SOUL.md`). | Remediated |
|
||||
| MF-007 | Medium | Marketing skills used `.claude/product-marketing-context.md` instead of Mosaic repo-local path. | Remediated |
|
||||
| MF-008 | Low | `agent-skills/README.md` manual install/adaptation guidance referenced legacy Claude-only paths. | Remediated |
|
||||
| MF-009 | Low | Historical creation-log contained stale Claude-specific path examples. | Remediated |
|
||||
|
||||
## Remediation Details
|
||||
|
||||
### MF-001 (QA rails path correction)
|
||||
|
||||
Updated:
|
||||
|
||||
- `tools/qa/qa-hook-wrapper.sh`
|
||||
- `tools/qa/qa-hook-stdin.sh`
|
||||
- `tools/qa/qa-hook-handler.sh`
|
||||
- `tools/qa/remediation-hook-handler.sh`
|
||||
- `tools/qa/qa-queue-monitor.sh`
|
||||
|
||||
Change:
|
||||
|
||||
- Standardized handler paths to `~/.config/mosaic/tools/qa/...`.
|
||||
|
||||
### MF-002 + MF-003 (conditional loading/context detection)
|
||||
|
||||
Updated:
|
||||
|
||||
- `tools/bootstrap/agent-lint.sh`
|
||||
- `tools/bootstrap/agent-upgrade.sh`
|
||||
- `templates/agent/SPEC.md`
|
||||
|
||||
Change:
|
||||
|
||||
- Detection now recognizes both legacy and Mosaic patterns:
|
||||
- `agent-guides` (legacy compatibility)
|
||||
- `~/.config/mosaic/guides`
|
||||
- `Conditional Loading`
|
||||
- `Conditional Context`
|
||||
|
||||
### MF-004 (runtime overlay drift-proofing)
|
||||
|
||||
Updated:
|
||||
|
||||
- `runtime/claude/CLAUDE.md`
|
||||
|
||||
Change:
|
||||
|
||||
- Removed hardcoded skills count text.
|
||||
|
||||
### MF-005 (pr-reviewer skill path migration)
|
||||
|
||||
Updated:
|
||||
|
||||
- `skills/pr-reviewer/SKILL.md`
|
||||
|
||||
Change:
|
||||
|
||||
- Replaced all `~/.claude/scripts/git/...` with `~/.config/mosaic/tools/git/...`.
|
||||
- Replaced `~/.claude/skills/...` with `~/.config/mosaic/skills/...`.
|
||||
|
||||
### MF-006 (worktree skill docs hierarchy)
|
||||
|
||||
Updated:
|
||||
|
||||
- `skills/using-git-worktrees/SKILL.md`
|
||||
|
||||
Change:
|
||||
|
||||
- Replaced CLAUDE-only checks with `AGENTS.md` / `SOUL.md` (fallback: `CLAUDE.md`).
|
||||
|
||||
### MF-007 (marketing context path migration)
|
||||
|
||||
Updated:
|
||||
|
||||
- `skills/product-marketing-context/SKILL.md`
|
||||
- All skills referencing product-marketing-context path (bulk update)
|
||||
|
||||
Change:
|
||||
|
||||
- Standardized context path to `.mosaic/product-marketing-context.md`.
|
||||
|
||||
### MF-008 + MF-009 (repo guidance cleanup)
|
||||
|
||||
Updated:
|
||||
|
||||
- `README.md` (agent-skills)
|
||||
- `skills/writing-skills/SKILL.md`
|
||||
- `skills/writing-skills/examples/CLAUDE_MD_TESTING.md`
|
||||
- `skills/systematic-debugging/CREATION-LOG.md`
|
||||
|
||||
Change:
|
||||
|
||||
- Standardized references to `~/.config/mosaic/skills`.
|
||||
- Removed stale Claude-only wording.
|
||||
|
||||
## Residual Legacy References (Intentional)
|
||||
|
||||
The following legacy references remain in `mosaic-bootstrap` by design and are not inconsistencies:
|
||||
|
||||
- Runtime compatibility tooling for Claude home directory management:
|
||||
- `bin/mosaic-link-runtime-assets`
|
||||
- `bin/mosaic-clean-runtime`
|
||||
- `bin/mosaic-doctor`
|
||||
- `bin/mosaic-sync-skills`
|
||||
- `bin/mosaic-migrate-local-skills`
|
||||
- Documentation explaining Claude runtime overlays:
|
||||
- `README.md`
|
||||
- `profiles/README.md`
|
||||
- `adapters/claude.md`
|
||||
- `runtime/claude/settings-overlays/` (sample overlay; now shipped sanitized under `examples/overlays/`)
|
||||
|
||||
These are required to support existing Claude runtime integration while keeping Mosaic as canonical source.
|
||||
|
||||
## Verification
|
||||
|
||||
Executed checks:
|
||||
|
||||
- `rg -n "~/.claude|\\.claude/|agent-guides" ~/src/agent-skills -S`
|
||||
- Result: no matches after remediation.
|
||||
- `rg -n "~/.config/mosaic/tools/(qa-hook|remediation-hook|qa-queue-monitor)" ~/src/mosaic-bootstrap -S`
|
||||
- Result: no invalid old-style QA rail paths remain.
|
||||
- Installed runtime validation:
|
||||
- `~/.config/mosaic` contains `tools/git`, `tools/portainer`, `tools/cicd`, `skills`, and `bin` tooling.
|
||||
@@ -0,0 +1,231 @@
|
||||
# Brief: Monorepo Consolidation — mosaic/stack → mosaic/mosaic-stack
|
||||
|
||||
## Source
|
||||
|
||||
Architecture consolidation — merge the mosaic/stack repo (Forge pipeline, MACP protocol, framework tools) into mosaic/mosaic-stack (Harness Foundation platform). Two repos doing related work that need to converge.
|
||||
|
||||
## Context
|
||||
|
||||
**mosaic/stack** (OLD) contains:
|
||||
|
||||
- Forge progressive refinement pipeline (stages, agents, personas, rails, debate protocol, brief classification)
|
||||
- MACP protocol (JSON schemas, deterministic Python controller, dispatcher, event system, gate runner)
|
||||
- Credential resolver (Python — OC config, mosaic files, ambient env, JSON5 parser)
|
||||
- OC framework plugin (injects Mosaic rails into all agent sessions)
|
||||
- Profiles (runtime-neutral context packs for tech stacks and domains)
|
||||
- Stage adapter (Forge→MACP bridge)
|
||||
- Board tasks (multi-agent board evaluation)
|
||||
- OpenBrain specialist memory (learning capture/recall)
|
||||
- 17 guides, 5 universal skills
|
||||
|
||||
**mosaic/mosaic-stack** (NEW) contains:
|
||||
|
||||
- Harness Foundation platform (NestJS gateway, Next.js web, Drizzle ORM, Pi SDK runtime)
|
||||
- 5 provider adapters, task classifier, routing rules, model capability matrix
|
||||
- MACP OC plugin (ACP runtime backend with Pi bridge)
|
||||
- TS coord package (mission runner, tasks file manager, status tracker — 1635 lines)
|
||||
- BullMQ job queue, OTEL telemetry, channel plugins (Discord, Telegram)
|
||||
- CLI with TUI, 65/65 tasks done, v0.2.0
|
||||
|
||||
**Decision:** NEW repo is the base. All unique work from OLD gets ported into NEW as packages.
|
||||
|
||||
## Scope
|
||||
|
||||
### Work Package 1: Forge Pipeline Package (`packages/forge`)
|
||||
|
||||
Port the entire Forge progressive refinement pipeline as a TypeScript package.
|
||||
|
||||
**From OLD:**
|
||||
|
||||
- `forge/pipeline/stages/*.md` — 11 stage definitions
|
||||
- `forge/pipeline/agents/{board,generalists,specialists,cross-cutting}/*.md` — all persona definitions
|
||||
- `forge/pipeline/rails/*.md` — debate protocol, dynamic composition, worker rails
|
||||
- `forge/pipeline/gates/` — gate reviewer definitions
|
||||
- `forge/pipeline/orchestrator/run-structure.md` — file-based observability spec
|
||||
- `forge/templates/` — brief and PRD templates
|
||||
- `forge/pipeline/orchestrator/board_tasks.py` → rewrite in TS
|
||||
- `forge/pipeline/orchestrator/stage_adapter.py` → rewrite in TS
|
||||
- `forge/pipeline/orchestrator/pipeline_runner.py` → rewrite in TS
|
||||
- `forge/forge` CLI (Python) → rewrite in TS, integrate with `packages/cli`
|
||||
|
||||
**Package structure:**
|
||||
|
||||
```
|
||||
packages/forge/
|
||||
├── src/
|
||||
│ ├── index.ts # Public API
|
||||
│ ├── pipeline-runner.ts # Orchestrates full pipeline run
|
||||
│ ├── stage-adapter.ts # Maps stages to MACP/coord tasks
|
||||
│ ├── board-tasks.ts # Multi-agent board evaluation task generator
|
||||
│ ├── brief-classifier.ts # strategic/technical/hotfix classification
|
||||
│ ├── types.ts # Stage specs, run manifest, gate results
|
||||
│ └── constants.ts # Stage sequence, timeouts, labels
|
||||
├── pipeline/
|
||||
│ ├── stages/ # .md stage definitions (copied)
|
||||
│ ├── agents/ # .md persona definitions (copied)
|
||||
│ │ ├── board/
|
||||
│ │ ├── cross-cutting/
|
||||
│ │ ├── generalists/
|
||||
│ │ └── specialists/
|
||||
│ │ ├── language/
|
||||
│ │ └── domain/
|
||||
│ ├── rails/ # .md rails (copied)
|
||||
│ ├── gates/ # .md gate definitions (copied)
|
||||
│ └── templates/ # brief + PRD templates (copied)
|
||||
└── package.json
|
||||
```
|
||||
|
||||
**Key design decisions:**
|
||||
|
||||
- Pipeline markdown assets are runtime data, not compiled — ship as-is in the package
|
||||
- `pipeline-runner.ts` calls into `packages/coord` for task execution (not a separate controller)
|
||||
- Stage adapter generates coord-compatible tasks, not MACP JSON directly
|
||||
- Board tasks use `depends_on_policy: "all_terminal"` for synthesis
|
||||
- Per-stage timeouts from `STAGE_TIMEOUTS` map
|
||||
- Brief classifier supports CLI flag, YAML frontmatter, and keyword auto-detection
|
||||
- Run output goes to project-scoped `.forge/runs/{run-id}/` (not inside the Forge package)
|
||||
|
||||
**Persona override system (new):**
|
||||
|
||||
- Base personas ship with the package (read-only)
|
||||
- Project-level overrides in `.forge/personas/{role}.md` extend (not replace) base personas
|
||||
- Board composition configurable via `.forge/config.yaml`:
|
||||
```yaml
|
||||
board:
|
||||
additional_members:
|
||||
- compliance-officer.md
|
||||
skip_members: []
|
||||
specialists:
|
||||
always_include:
|
||||
- proxmox-expert
|
||||
```
|
||||
- OpenBrain integration for cross-run specialist memory (when enabled)
|
||||
|
||||
### Work Package 2: MACP Protocol Package (`packages/macp`)
|
||||
|
||||
Port the MACP protocol layer, event system, and gate runner as a TypeScript package.
|
||||
|
||||
**From OLD:**
|
||||
|
||||
- `tools/macp/protocol/task.schema.json` — task JSON schema
|
||||
- `tools/macp/protocol/` — event schemas
|
||||
- `tools/macp/controller/gate_runner.py` → rewrite in TS as `gate-runner.ts`
|
||||
- `tools/macp/events/` — event watcher, webhook adapter, Discord formatter → rewrite in TS
|
||||
- `tools/macp/dispatcher/credential_resolver.py` → rewrite in TS as `credential-resolver.ts`
|
||||
- `tools/macp/memory/learning_capture.py` + `learning_recall.py` → rewrite in TS
|
||||
|
||||
**Package structure:**
|
||||
|
||||
```
|
||||
packages/macp/
|
||||
├── src/
|
||||
│ ├── index.ts # Public API
|
||||
│ ├── types.ts # Task, event, result, gate types
|
||||
│ ├── schemas/ # JSON schemas (copied)
|
||||
│ ├── gate-runner.ts # Mechanical + AI review quality gates
|
||||
│ ├── credential-resolver.ts # Provider credential resolution (mosaic files, OC config, ambient)
|
||||
│ ├── event-emitter.ts # Append events to ndjson, structured event types
|
||||
│ ├── event-watcher.ts # Poll events.ndjson with cursor persistence
|
||||
│ ├── webhook-adapter.ts # POST events to configurable URL
|
||||
│ ├── discord-formatter.ts # Human-readable event messages
|
||||
│ └── learning.ts # OpenBrain capture + recall
|
||||
└── package.json
|
||||
```
|
||||
|
||||
**Integration with existing packages:**
|
||||
|
||||
- `packages/coord` uses `packages/macp` for event emission, gate running, and credential resolution
|
||||
- `plugins/macp` uses `packages/macp` for protocol types and credential resolution
|
||||
- `packages/forge` uses `packages/macp` gate types for stage gates
|
||||
|
||||
### Work Package 3: OC Framework Plugin (`plugins/mosaic-framework`)
|
||||
|
||||
Port the OC framework plugin that injects Mosaic rails into all agent sessions.
|
||||
|
||||
**From OLD:**
|
||||
|
||||
- `oc-plugins/mosaic-framework/index.ts` — `before_agent_start` + `subagent_spawning` hooks
|
||||
- `oc-plugins/mosaic-framework/openclaw.plugin.json`
|
||||
|
||||
**Structure:**
|
||||
|
||||
```
|
||||
plugins/mosaic-framework/
|
||||
├── src/
|
||||
│ └── index.ts # Plugin hooks
|
||||
└── package.json
|
||||
```
|
||||
|
||||
**This is separate from `plugins/macp`:**
|
||||
|
||||
- `mosaic-framework` = injects Mosaic rails/contracts into every OC session (passive enforcement)
|
||||
- `macp` = provides an ACP runtime backend for MACP task execution (active runtime)
|
||||
|
||||
### Work Package 4: Profiles + Guides + Skills
|
||||
|
||||
Port reference content as a documentation/config package or top-level directories.
|
||||
|
||||
**From OLD:**
|
||||
|
||||
- `profiles/domains/*.json` — HIPAA, fintech, crypto context packs
|
||||
- `profiles/tech-stacks/*.json` — NestJS, Next.js, FastAPI, React conventions
|
||||
- `profiles/workflows/*.json` — API development, frontend component, testing workflows
|
||||
- `guides/*.md` — 17 guides (auth, backend, QA, orchestrator, PRD, etc.)
|
||||
- `skills-universal/` — jarvis, macp, mosaic-standards, prd, setup-cicd skills
|
||||
|
||||
**Destination:**
|
||||
|
||||
```
|
||||
profiles/ # Top-level (same as OLD)
|
||||
guides/ # Top-level (same as OLD)
|
||||
skills/ # Top-level (renamed from skills-universal)
|
||||
```
|
||||
|
||||
These are runtime-neutral assets consumed by any agent or profile loader — they don't belong in a compiled package.
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Rewriting the NestJS orchestrator app from OLD (`apps/orchestrator/`) — its functionality is subsumed by `packages/coord` + `apps/gateway`
|
||||
- Porting the FastAPI coordinator from OLD (`apps/coordinator/`) — its functionality (webhook receiver, issue parser, quality orchestrator) is handled by `packages/coord` + `apps/gateway` in the new architecture
|
||||
- Porting the Prisma schema or OLD's `apps/api` — Drizzle migration is complete
|
||||
- Old Docker Compose configs (Traefik, Matrix, OpenBao) — NEW has its own infra setup
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. `packages/forge` exists with all 11 stage definitions, all persona markdowns, all rails, and TS implementations of pipeline-runner, stage-adapter, board-tasks, and brief-classifier
|
||||
2. `packages/macp` exists with gate-runner, credential-resolver, event system, and learning capture/recall — all in TypeScript
|
||||
3. `plugins/mosaic-framework` exists and registers OC hooks for rails injection
|
||||
4. Profiles, guides, and skills are present at top-level
|
||||
5. `packages/forge` integrates with `packages/coord` for task execution
|
||||
6. `packages/macp` credential-resolver is used by `plugins/macp` Pi bridge
|
||||
7. All existing tests pass (no regressions)
|
||||
8. New packages have test coverage ≥85%
|
||||
9. `pnpm lint && pnpm typecheck && pnpm build` passes
|
||||
10. `.forge/runs/` project-scoped output directory works for at least one test run
|
||||
|
||||
## Technical Constraints
|
||||
|
||||
- All new code is ESM with NodeNext module resolution
|
||||
- No Python in the new repo — everything rewrites to TypeScript
|
||||
- Pipeline markdown assets (stages, personas, rails) are shipped as package data, not compiled
|
||||
- Credential resolver must support: mosaic credential files, OC config (JSON5), ambient environment — same resolution order as the Python version
|
||||
- Must preserve `depends_on_policy` semantics (all, any, all_terminal)
|
||||
- Per-stage timeouts must be preserved
|
||||
- JSON5 stripping must use the placeholder-extraction approach (not naive regex on string content)
|
||||
|
||||
## Estimated Complexity
|
||||
|
||||
High — crosses 4 work packages with protocol porting, TS rewrites, and integration wiring. Each work package is independently shippable.
|
||||
|
||||
**Suggested execution order:**
|
||||
|
||||
1. WP4 (profiles/guides/skills) — pure copy, no code, fast win
|
||||
2. WP2 (packages/macp) — protocol foundation, needed by WP1 and WP3
|
||||
3. WP1 (packages/forge) — the big one, depends on WP2
|
||||
4. WP3 (plugins/mosaic-framework) — OC integration, can parallel with WP1
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `packages/coord` must be stable (it is — WP1 integrates with it)
|
||||
- `plugins/macp` must be stable (it is — WP2 provides types/credentials to it)
|
||||
- Pi SDK (`@mariozechner/pi-agent-core`) already in the dependency tree
|
||||
@@ -0,0 +1,228 @@
|
||||
#!/usr/bin/env python3
|
||||
"""P5 Gate0 replay probe; BUILT ONLY, execution is Mos-gated.
|
||||
|
||||
Run only under fresh-executor authorization:
|
||||
python3 -I -S -B docs/compaction-refresh/probes/p5_receipt_replay.py
|
||||
|
||||
Each of the default three isolated runs launches the shipped lease-broker daemon
|
||||
in a distinct private temporary directory. This driver never changes broker
|
||||
state directly and does not replace the promote gate: every transition is sent
|
||||
over the daemon's real Unix socket. It proves the shipped order is
|
||||
PENDING_DELIVERY -> observe/evidence commit -> consume -> VERIFIED and that a
|
||||
consumed challenge cannot be replayed or reopen/renew its lease.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
REPOSITORY = HERE.parents[2]
|
||||
TOOLS = REPOSITORY / "packages/mosaic/framework/tools/lease-broker"
|
||||
DAEMON = TOOLS / "daemon.py"
|
||||
FRAGMENTS = TOOLS / "normative_fragments.py"
|
||||
|
||||
|
||||
def load_shipped_fragments():
|
||||
if not FRAGMENTS.is_file():
|
||||
raise RuntimeError(f"shipped normative construction missing: {FRAGMENTS}")
|
||||
spec = importlib.util.spec_from_file_location("p5_shipped_normative_fragments", FRAGMENTS)
|
||||
if spec is None or spec.loader is None:
|
||||
raise RuntimeError("unable to load shipped normative construction")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def request(socket_path: Path, value: dict[str, object]) -> dict[str, object]:
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as connection:
|
||||
connection.settimeout(3.0)
|
||||
connection.connect(str(socket_path))
|
||||
connection.sendall((json.dumps(value, separators=(",", ":")) + "\n").encode())
|
||||
connection.shutdown(socket.SHUT_WR)
|
||||
response = bytearray()
|
||||
while True:
|
||||
chunk = connection.recv(4096)
|
||||
if not chunk:
|
||||
break
|
||||
response.extend(chunk)
|
||||
if not response.endswith(b"\n") or response.count(b"\n") != 1:
|
||||
raise AssertionError(f"unframed broker reply: {bytes(response)!r}")
|
||||
parsed = json.loads(response[:-1])
|
||||
if not isinstance(parsed, dict):
|
||||
raise AssertionError(f"non-object broker reply: {parsed!r}")
|
||||
return parsed
|
||||
|
||||
|
||||
def wait_ready(process: subprocess.Popen[str], socket_path: Path) -> None:
|
||||
deadline = time.monotonic() + 5.0
|
||||
while time.monotonic() < deadline:
|
||||
if socket_path.exists():
|
||||
return
|
||||
if process.poll() is not None:
|
||||
output = process.stdout.read() if process.stdout is not None else ""
|
||||
raise RuntimeError(f"shipped daemon exited before READY: {output}")
|
||||
time.sleep(0.02)
|
||||
raise TimeoutError("shipped daemon did not create private probe socket")
|
||||
|
||||
|
||||
def expect_refused(reply: dict[str, object], code: str) -> None:
|
||||
if reply != {"ok": False, "code": code}:
|
||||
raise AssertionError(f"expected refusal {code}, got {reply!r}")
|
||||
|
||||
|
||||
def run_once(index: int) -> str:
|
||||
fragments = load_shipped_fragments()
|
||||
root = Path(tempfile.mkdtemp(prefix=f"mosaic-p5-replay-{index}-"))
|
||||
os.chmod(root, 0o700)
|
||||
socket_path = root / "broker.sock"
|
||||
state_path = root / "state.json"
|
||||
observer_path = root / "test-observer.json"
|
||||
process = subprocess.Popen(
|
||||
[
|
||||
sys.executable, "-I", "-S", "-B", str(DAEMON), "--socket", str(socket_path),
|
||||
"--state", str(state_path), "--test-observer-file", str(observer_path),
|
||||
],
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
)
|
||||
try:
|
||||
wait_ready(process, socket_path)
|
||||
registered = request(socket_path, {"action": "register_anchor", "runtime_generation": 1})
|
||||
if registered.get("ok") is not True or not isinstance(registered.get("session_id"), str):
|
||||
raise AssertionError(f"registration failed: {registered!r}")
|
||||
session_id = registered["session_id"]
|
||||
construction = fragments.build_payload(
|
||||
manifest_version=1,
|
||||
generator_version="p5-replay-probe",
|
||||
fragments=[
|
||||
fragments.NormativeFragment(
|
||||
"authority/probe",
|
||||
b"P5 shipped transition driver\n",
|
||||
"63537df1a6cb0d80195a96757ab11d629e5b5e1f23be167218b84cb195b1c1d6",
|
||||
),
|
||||
],
|
||||
)
|
||||
if construction.injectionDecision != "ACCEPTED" or not construction.promotion:
|
||||
raise AssertionError("shipped normative construction refused P5 fixture")
|
||||
binding = {
|
||||
"compaction_epoch": index,
|
||||
"request_epoch": index + 100,
|
||||
"h_source": construction.h_source,
|
||||
"h_payload": construction.h_payload,
|
||||
"schema_version": 1,
|
||||
}
|
||||
construction_request = {
|
||||
"manifest_version": 1,
|
||||
"generator_version": "p5-replay-probe",
|
||||
"fragments": [{
|
||||
"source_id": "authority/probe",
|
||||
"content_base64": base64.b64encode(b"P5 shipped transition driver\n").decode("ascii"),
|
||||
"expected_sha256": "63537df1a6cb0d80195a96757ab11d629e5b5e1f23be167218b84cb195b1c1d6",
|
||||
}],
|
||||
}
|
||||
pending = request(socket_path, {
|
||||
"action": "begin_verification",
|
||||
"session_id": session_id,
|
||||
"runtime_generation": 1,
|
||||
"runtime": "pi",
|
||||
"binding": binding,
|
||||
"construction": construction_request,
|
||||
})
|
||||
if pending.get("ok") is not True or pending.get("state") != "PENDING_VERIFICATION":
|
||||
raise AssertionError(f"shipped pending-delivery transition failed: {pending!r}")
|
||||
challenge = pending.get("receipt_challenge")
|
||||
receipt = pending.get("receipt")
|
||||
if not isinstance(challenge, str) or not isinstance(receipt, str):
|
||||
raise AssertionError(f"shipped broker did not mint a receipt challenge: {pending!r}")
|
||||
|
||||
# Promotion before observation/evidence/consumption is forbidden.
|
||||
expect_refused(request(socket_path, {
|
||||
"action": "promote_lease",
|
||||
"session_id": session_id,
|
||||
"runtime_generation": 1,
|
||||
"receipt_challenge": challenge,
|
||||
}), "INVALID_LEASE_TRANSITION")
|
||||
|
||||
observer_path.write_text(json.dumps({
|
||||
"session_id": session_id,
|
||||
"runtime_generation": 1,
|
||||
"latest_assistant_message": receipt,
|
||||
}), encoding="utf-8")
|
||||
os.chmod(observer_path, 0o600)
|
||||
observed = request(socket_path, {
|
||||
"action": "observe_receipt",
|
||||
"session_id": session_id,
|
||||
"runtime_generation": 1,
|
||||
"receipt_challenge": challenge,
|
||||
})
|
||||
if observed.get("ok") is not True or observed.get("state") != "PENDING_PROMOTION":
|
||||
raise AssertionError(f"shipped evidence transition failed: {observed!r}")
|
||||
durable = json.loads(state_path.read_text(encoding="utf-8"))
|
||||
evidence = durable["tokens"][challenge].get("evidence")
|
||||
if not isinstance(evidence, dict) or not isinstance(evidence.get("h_latest_assistant"), str):
|
||||
raise AssertionError("shipped receipt evidence was not committed before consume/promote")
|
||||
|
||||
promoted = request(socket_path, {
|
||||
"action": "promote_lease",
|
||||
"session_id": session_id,
|
||||
"runtime_generation": 1,
|
||||
"receipt_challenge": challenge,
|
||||
})
|
||||
if promoted.get("ok") is not True or promoted.get("state") != "VERIFIED":
|
||||
raise AssertionError(f"shipped consume-before-promote transition failed: {promoted!r}")
|
||||
|
||||
# T25/T28: the actual consumed challenge, re-presented through the
|
||||
# shipped daemon, can neither be observed again nor re-promote/reopen.
|
||||
expect_refused(request(socket_path, {
|
||||
"action": "observe_receipt",
|
||||
"session_id": session_id,
|
||||
"runtime_generation": 1,
|
||||
"receipt_challenge": challenge,
|
||||
}), "RECEIPT_REPLAY")
|
||||
expect_refused(request(socket_path, {
|
||||
"action": "promote_lease",
|
||||
"session_id": session_id,
|
||||
"runtime_generation": 1,
|
||||
"receipt_challenge": challenge,
|
||||
}), "RECEIPT_REPLAY")
|
||||
return challenge
|
||||
finally:
|
||||
if process.poll() is None:
|
||||
process.terminate()
|
||||
try:
|
||||
process.wait(timeout=3.0)
|
||||
except subprocess.TimeoutExpired:
|
||||
process.kill()
|
||||
process.wait()
|
||||
shutil.rmtree(root, ignore_errors=True)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--runs", type=int, default=3)
|
||||
arguments = parser.parse_args()
|
||||
if arguments.runs != 3:
|
||||
raise SystemExit("P5 requires exactly three isolated runs")
|
||||
challenges = [run_once(index) for index in range(arguments.runs)]
|
||||
if len(set(challenges)) != arguments.runs:
|
||||
raise AssertionError("separate shipped cycles did not mint unique challenges")
|
||||
print("P5 receipt replay probe PASS: 3 isolated shipped-daemon runs")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,255 @@
|
||||
#!/usr/bin/env python3
|
||||
"""P6 constrained-recovery probe; BUILT ONLY and Mos-gated.
|
||||
|
||||
DO NOT self-fire. Under Mos authorization only:
|
||||
python3 -I -S -B docs/compaction-refresh/probes/p6_constrained_recovery.py
|
||||
|
||||
The default three isolated runs launch the shipped daemon plus its production
|
||||
observer transport on private sockets. The driver invokes the shipped recovery
|
||||
command and adapter gate identity; it never resets broker state, mocks promote,
|
||||
or taps a live model-output stream.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
REPOSITORY = HERE.parents[2]
|
||||
TOOLS = REPOSITORY / "packages/mosaic/framework/tools/lease-broker"
|
||||
DAEMON = TOOLS / "daemon.py"
|
||||
GATE = TOOLS / "mutator-gate.py"
|
||||
RECOVERY_COMMAND = TOOLS / "recover-context.py"
|
||||
OBSERVER_CLIENT = TOOLS / "receipt-observer-client.py"
|
||||
FRAGMENTS = TOOLS / "normative_fragments.py"
|
||||
CLAUDE_SETTINGS = REPOSITORY / "packages/mosaic/framework/runtime/claude/settings.json"
|
||||
PI_EXTENSION = REPOSITORY / "packages/mosaic/framework/runtime/pi/mosaic-extension.ts"
|
||||
|
||||
|
||||
def load_shipped_fragments():
|
||||
spec = importlib.util.spec_from_file_location("p6_shipped_fragments", FRAGMENTS)
|
||||
if spec is None or spec.loader is None:
|
||||
raise RuntimeError("shipped normative construction unavailable")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def request(socket_path: Path, value: dict[str, object]) -> dict[str, object]:
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as connection:
|
||||
connection.settimeout(3.0)
|
||||
connection.connect(str(socket_path))
|
||||
connection.sendall((json.dumps(value, separators=(",", ":")) + "\n").encode())
|
||||
connection.shutdown(socket.SHUT_WR)
|
||||
response = bytearray()
|
||||
while True:
|
||||
chunk = connection.recv(4096)
|
||||
if not chunk:
|
||||
break
|
||||
response.extend(chunk)
|
||||
if not response.endswith(b"\n") or response.count(b"\n") != 1:
|
||||
raise AssertionError(f"unframed broker reply: {bytes(response)!r}")
|
||||
reply = json.loads(response[:-1])
|
||||
if not isinstance(reply, dict):
|
||||
raise AssertionError("broker reply is not an object")
|
||||
return reply
|
||||
|
||||
|
||||
def wait_ready(process: subprocess.Popen[str], socket_path: Path) -> None:
|
||||
deadline = time.monotonic() + 5.0
|
||||
while time.monotonic() < deadline:
|
||||
if socket_path.exists():
|
||||
return
|
||||
if process.poll() is not None:
|
||||
output = process.stdout.read() if process.stdout is not None else ""
|
||||
raise RuntimeError(f"shipped daemon exited before READY: {output}")
|
||||
time.sleep(0.02)
|
||||
raise TimeoutError("shipped daemon did not create private probe socket")
|
||||
|
||||
|
||||
def run_json(command: list[str], environment: dict[str, str], input_value: object | None = None) -> dict[str, object]:
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
input=None if input_value is None else json.dumps(input_value),
|
||||
text=True,
|
||||
capture_output=True,
|
||||
env=environment,
|
||||
check=False,
|
||||
)
|
||||
if not completed.stdout.endswith("\n"):
|
||||
raise AssertionError(f"command omitted framed result: {completed.stderr!r}")
|
||||
reply = json.loads(completed.stdout)
|
||||
if not isinstance(reply, dict):
|
||||
raise AssertionError("command result is not an object")
|
||||
return reply
|
||||
|
||||
|
||||
def gate_recovery(runtime: str, phase: str, environment: dict[str, str]) -> None:
|
||||
command = [sys.executable, "-I", "-S", "-B", str(GATE), "--runtime", runtime]
|
||||
if runtime == "claude":
|
||||
command.extend(["--recovery-command", str(RECOVERY_COMMAND)])
|
||||
recovery_invocation = (
|
||||
f"python3 {RECOVERY_COMMAND} begin --construction /tmp/p6.json "
|
||||
"--compaction-epoch 1 --request-epoch 1"
|
||||
if phase == "begin"
|
||||
else f"python3 {RECOVERY_COMMAND} complete"
|
||||
)
|
||||
value = {"tool_name": "Bash", "tool_input": {"command": recovery_invocation}}
|
||||
else:
|
||||
value = {"tool_name": "mosaic_context_recover"}
|
||||
completed = subprocess.run(command, input=json.dumps(value), text=True, capture_output=True, env=environment, check=False)
|
||||
if completed.returncode != 0:
|
||||
raise AssertionError(f"{runtime} recovery invocation remained gated: {completed.stderr!r}")
|
||||
|
||||
|
||||
def record_production_observation(runtime: str, message: str, root: Path, environment: dict[str, str]) -> None:
|
||||
command = [sys.executable, "-I", "-S", "-B", str(OBSERVER_CLIENT), "--runtime", runtime]
|
||||
if runtime == "claude":
|
||||
transcript = root / "claude-transcript.jsonl"
|
||||
transcript.write_text(json.dumps({"message": {"role": "assistant", "content": message}}) + "\n", encoding="utf-8")
|
||||
payload = {"transcript_path": str(transcript)}
|
||||
command.append("--latest-entry")
|
||||
else:
|
||||
payload = {"latest_assistant_message": message}
|
||||
completed = subprocess.run(command, input=json.dumps(payload), text=True, capture_output=True, env=environment, check=False)
|
||||
if completed.returncode != 0:
|
||||
raise AssertionError(f"{runtime} production observer transport refused: {completed.stderr!r}")
|
||||
|
||||
|
||||
def run_once(index: int, runtime: str) -> None:
|
||||
# Parity guard: drive the shipped command and the repaired adapter/observer
|
||||
# bytes, not a shadow receipt or promotion implementation.
|
||||
recovery_source = RECOVERY_COMMAND.read_text(encoding="utf-8")
|
||||
if '"action": "begin_recovery"' not in recovery_source or '"action": "complete_recovery"' not in recovery_source:
|
||||
raise AssertionError("P6 parity guard: recovery command no longer drives shipped broker entrypoints")
|
||||
gate_source = GATE.read_text(encoding="utf-8")
|
||||
if "--recovery-command" not in CLAUDE_SETTINGS.read_text(encoding="utf-8"):
|
||||
raise AssertionError("P6 parity guard: Claude recovery mapping is missing")
|
||||
if "_SHELL_ACTIVE" not in gate_source or "argv[1] != str(recovery_command)" not in gate_source:
|
||||
raise AssertionError("P6 parity guard: Claude mapping is not literal-only")
|
||||
if "const RECOVERY_TOOL = 'mosaic_context_recover'" not in PI_EXTENSION.read_text(encoding="utf-8"):
|
||||
raise AssertionError("P6 parity guard: Pi recovery tool mapping is missing")
|
||||
|
||||
fragments = load_shipped_fragments()
|
||||
root = Path(tempfile.mkdtemp(prefix=f"mosaic-p6-recovery-{index}-"))
|
||||
os.chmod(root, 0o700)
|
||||
socket_path = root / "broker.sock"
|
||||
observer_socket = root / "observer.sock"
|
||||
state_path = root / "state.json"
|
||||
construction_path = root / "construction.json"
|
||||
content = b"P6 constrained recovery fixture\n"
|
||||
construction = {
|
||||
"manifest_version": 1,
|
||||
"generator_version": "p6-constrained-recovery",
|
||||
"fragments": [{
|
||||
"source_id": "authority/p6",
|
||||
"content_base64": base64.b64encode(content).decode("ascii"),
|
||||
"expected_sha256": hashlib.sha256(content).hexdigest(),
|
||||
}],
|
||||
}
|
||||
construction_path.write_text(json.dumps(construction), encoding="utf-8")
|
||||
os.chmod(construction_path, 0o600)
|
||||
process = subprocess.Popen(
|
||||
[sys.executable, "-I", "-S", "-B", str(DAEMON), "--socket", str(socket_path),
|
||||
"--state", str(state_path), "--observer-socket", str(observer_socket)],
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
)
|
||||
try:
|
||||
wait_ready(process, socket_path)
|
||||
registered = request(socket_path, {"action": "register_anchor", "runtime_generation": 1})
|
||||
session_id = registered.get("session_id")
|
||||
if registered.get("ok") is not True or not isinstance(session_id, str):
|
||||
raise AssertionError(f"broker anchor registration failed: {registered!r}")
|
||||
built = fragments.build_payload_from_wire(construction)
|
||||
normal = request(socket_path, {
|
||||
"action": "begin_verification", "session_id": session_id, "runtime_generation": 1,
|
||||
"runtime": runtime, "construction": construction,
|
||||
"binding": {"compaction_epoch": index, "request_epoch": index + 100,
|
||||
"h_source": built.h_source, "h_payload": built.h_payload, "schema_version": 1},
|
||||
})
|
||||
normal_challenge = normal.get("receipt_challenge")
|
||||
normal_receipt = normal.get("receipt")
|
||||
if not isinstance(normal_challenge, str) or not isinstance(normal_receipt, str):
|
||||
raise AssertionError("normal path did not mint a receipt challenge")
|
||||
environment = {
|
||||
**os.environ,
|
||||
"MOSAIC_LEASE_BROKER_SOCKET": str(socket_path),
|
||||
"MOSAIC_RECEIPT_OBSERVER_SOCKET": str(observer_socket),
|
||||
"MOSAIC_LEASE_SESSION_ID": session_id,
|
||||
"MOSAIC_RUNTIME_GENERATION": "1",
|
||||
"MOSAIC_LEASE_RUNTIME": runtime,
|
||||
}
|
||||
gate_recovery(runtime, "begin", environment)
|
||||
recovery = run_json([
|
||||
sys.executable, "-I", "-S", "-B", str(RECOVERY_COMMAND), "begin", "--construction", str(construction_path),
|
||||
"--compaction-epoch", str(index + 10), "--request-epoch", str(index + 110),
|
||||
], environment)
|
||||
challenge = recovery.get("receipt_challenge")
|
||||
receipt = recovery.get("receipt")
|
||||
if recovery.get("state") != "PENDING_DELIVERY" or not isinstance(challenge, str) or not isinstance(receipt, str):
|
||||
raise AssertionError(f"recovery command did not drive pending delivery: {recovery!r}")
|
||||
if challenge == normal_challenge:
|
||||
raise AssertionError("recovery reused a normal-path challenge")
|
||||
|
||||
# C4: production observer content is still exact-current-cycle only.
|
||||
record_production_observation(runtime, normal_receipt, root, environment)
|
||||
refused = run_json([sys.executable, "-I", "-S", "-B", str(RECOVERY_COMMAND), "complete"], environment)
|
||||
if refused.get("ok") is not False or refused.get("code") != "RECEIPT_MISMATCH":
|
||||
raise AssertionError(f"normal-path receipt replay was not refused: {refused!r}")
|
||||
|
||||
gate_recovery(runtime, "begin", environment)
|
||||
recovery = run_json([
|
||||
sys.executable, "-I", "-S", "-B", str(RECOVERY_COMMAND), "begin", "--construction", str(construction_path),
|
||||
"--compaction-epoch", str(index + 20), "--request-epoch", str(index + 120),
|
||||
], environment)
|
||||
receipt = recovery.get("receipt")
|
||||
if recovery.get("state") != "PENDING_DELIVERY" or not isinstance(receipt, str):
|
||||
raise AssertionError(f"fresh recovery retry did not pend: {recovery!r}")
|
||||
record_production_observation(runtime, receipt, root, environment)
|
||||
gate_recovery(runtime, "complete", environment)
|
||||
promoted = run_json([sys.executable, "-I", "-S", "-B", str(RECOVERY_COMMAND), "complete"], environment)
|
||||
if promoted.get("ok") is not True or promoted.get("state") != "VERIFIED":
|
||||
raise AssertionError(f"recovery consume-before-promote failed: {promoted!r}")
|
||||
replay = run_json([sys.executable, "-I", "-S", "-B", str(RECOVERY_COMMAND), "complete"], environment)
|
||||
if replay.get("ok") is not False or replay.get("code") != "INVALID_LEASE_TRANSITION":
|
||||
raise AssertionError(f"consumed recovery challenge re-promoted: {replay!r}")
|
||||
finally:
|
||||
if process.poll() is None:
|
||||
process.terminate()
|
||||
try:
|
||||
process.wait(timeout=3.0)
|
||||
except subprocess.TimeoutExpired:
|
||||
process.kill()
|
||||
process.wait()
|
||||
shutil.rmtree(root, ignore_errors=True)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--runs", type=int, default=3)
|
||||
arguments = parser.parse_args()
|
||||
if arguments.runs != 3:
|
||||
raise SystemExit("P6 requires exactly three isolated runs")
|
||||
for index, runtime in enumerate(("pi", "claude", "pi")):
|
||||
run_once(index, runtime)
|
||||
print("P6 constrained recovery probe PASS: 3 isolated shipped recovery-command runs")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,70 @@
|
||||
# deploy/portainer/
|
||||
|
||||
Portainer stack templates for Mosaic Stack deployments.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
| -------------------------- | -------------------------------------------------------------------------------------------------------------- |
|
||||
| `federated-test.stack.yml` | Docker Swarm stack for federation end-to-end test instances (`mos-test-1.woltje.com`, `mos-test-2.woltje.com`) |
|
||||
|
||||
---
|
||||
|
||||
## federated-test.stack.yml
|
||||
|
||||
A self-contained Swarm stack that boots a federated-tier Mosaic gateway with co-located Postgres 17 (pgvector) and Valkey 8. This is a **test template** — production deployments will use a separate template with stricter resource limits and Docker secrets.
|
||||
|
||||
### Deploy via Portainer UI
|
||||
|
||||
1. Log into Portainer.
|
||||
2. Navigate to **Stacks → Add stack**.
|
||||
3. Set a stack name matching `STACK_NAME` below (e.g. `mos-test-1`).
|
||||
4. Choose **Web editor** and paste the contents of `federated-test.stack.yml`.
|
||||
5. Scroll to **Environment variables** and add each variable listed below.
|
||||
6. Click **Deploy the stack**.
|
||||
|
||||
### Required environment variables
|
||||
|
||||
| Variable | Example | Notes |
|
||||
| -------------------- | --------------------------------------- | -------------------------------------------------------- |
|
||||
| `STACK_NAME` | `mos-test-1` | Unique per stack — used in Traefik router/service names. |
|
||||
| `HOST_FQDN` | `mos-test-1.woltje.com` | Fully-qualified hostname served by this stack. |
|
||||
| `POSTGRES_PASSWORD` | _(generate randomly)_ | Database password. Do **not** reuse between stacks. |
|
||||
| `BETTER_AUTH_SECRET` | _(generate: `openssl rand -base64 32`)_ | BetterAuth session signing key. |
|
||||
| `BETTER_AUTH_URL` | `https://mos-test-1.woltje.com` | Public base URL of the gateway. |
|
||||
|
||||
Optional variables (uncomment in the YAML or set in Portainer):
|
||||
|
||||
| Variable | Notes |
|
||||
| ----------------------------- | ---------------------------------------------------------- |
|
||||
| `ANTHROPIC_API_KEY` | Enable Claude models. |
|
||||
| `OPENAI_API_KEY` | Enable OpenAI models. |
|
||||
| `OTEL_EXPORTER_OTLP_ENDPOINT` | Forward traces to a collector (e.g. `http://jaeger:4318`). |
|
||||
|
||||
### Required external resources
|
||||
|
||||
Before deploying, ensure the following exist on the Swarm:
|
||||
|
||||
1. **`traefik-public` overlay network** — shared network Traefik uses to route traffic to stacks.
|
||||
```bash
|
||||
docker network create --driver overlay --attachable traefik-public
|
||||
```
|
||||
2. **`letsencrypt` cert resolver** — configured in the Traefik Swarm stack. The stack template references `tls.certresolver=letsencrypt`; the name must match your Traefik config.
|
||||
3. **DNS A record** — `${HOST_FQDN}` must resolve to the Swarm ingress IP (or a Cloudflare-proxied address pointing there).
|
||||
|
||||
### Deployed instances
|
||||
|
||||
| Stack name | HOST_FQDN | Purpose |
|
||||
| ------------ | ----------------------- | ---------------------------------- |
|
||||
| `mos-test-1` | `mos-test-1.woltje.com` | DEPLOY-03 — first federation peer |
|
||||
| `mos-test-2` | `mos-test-2.woltje.com` | DEPLOY-04 — second federation peer |
|
||||
|
||||
### Image
|
||||
|
||||
The gateway image is pinned by digest to `fed-v0.1.0-m1` (verified in DEPLOY-01). Update the digest in the YAML when promoting a new build — never use `:latest` or a mutable tag in Swarm.
|
||||
|
||||
### Notes
|
||||
|
||||
- This template boots a **vanilla M1-baseline gateway** in federated tier. Federation grants (Step-CA, mTLS) are M2+ scope and not included here.
|
||||
- Each stack gets its own Postgres volume (`postgres-data`) and Valkey volume (`valkey-data`) scoped to the stack name by Swarm.
|
||||
- `depends_on` is honoured by Compose but ignored by Swarm — healthchecks on Postgres and Valkey ensure the gateway retries until they are ready.
|
||||
@@ -0,0 +1,160 @@
|
||||
# deploy/portainer/federated-test.stack.yml
|
||||
#
|
||||
# Portainer / Docker Swarm stack template — federated-tier test instance
|
||||
#
|
||||
# PURPOSE
|
||||
# Deploys a single federated-tier Mosaic gateway with co-located Postgres
|
||||
# (pgvector) and Valkey for end-to-end federation testing. Intended for
|
||||
# mos-test-1.woltje.com and mos-test-2.woltje.com (DEPLOY-03/04).
|
||||
#
|
||||
# REQUIRED ENV VARS (set per-stack in Portainer → Stacks → Environment variables)
|
||||
# STACK_NAME Unique name for Traefik router/service labels.
|
||||
# Examples: mos-test-1, mos-test-2
|
||||
# HOST_FQDN Fully-qualified domain name served by this stack.
|
||||
# Examples: mos-test-1.woltje.com, mos-test-2.woltje.com
|
||||
# POSTGRES_PASSWORD Database password — set per stack; do NOT commit a default.
|
||||
# BETTER_AUTH_SECRET Random 32-char string for BetterAuth session signing.
|
||||
# Generate: openssl rand -base64 32
|
||||
# BETTER_AUTH_URL Public gateway base URL, e.g. https://mos-test-1.woltje.com
|
||||
#
|
||||
# OPTIONAL ENV VARS (uncomment and set in Portainer to enable features)
|
||||
# ANTHROPIC_API_KEY sk-ant-...
|
||||
# OPENAI_API_KEY sk-...
|
||||
# OTEL_EXPORTER_OTLP_ENDPOINT http://<collector>:4318
|
||||
# OTEL_SERVICE_NAME (default: mosaic-gateway)
|
||||
#
|
||||
# REQUIRED EXTERNAL RESOURCES
|
||||
# traefik-public Docker overlay network — must exist before deploying.
|
||||
# Create: docker network create --driver overlay --attachable traefik-public
|
||||
# letsencrypt Traefik cert resolver configured on the Swarm manager.
|
||||
# DNS A record ${HOST_FQDN} → Swarm ingress IP (or Cloudflare proxy).
|
||||
#
|
||||
# IMAGE
|
||||
# Pinned to sha-9f1a081 (main HEAD post-#488 Dockerfile fix). The previous
|
||||
# pin (fed-v0.1.0-m1, sha256:9b72e2...) had a broken pnpm copy and could
|
||||
# not resolve @mosaicstack/storage at runtime. The new digest was smoke-
|
||||
# tested locally — gateway boots, imports resolve, tier-detector runs.
|
||||
# Update digest here when promoting a new build.
|
||||
#
|
||||
# HEALTHCHECK NOTE (2026-04-21)
|
||||
# Switched from busybox wget to node http.get on 127.0.0.1 (not localhost) to
|
||||
# avoid IPv6 resolution issues on Alpine. Retries increased to 5 and
|
||||
# start_period to 60s to cover the NestJS/GC cold-start window (~40-50s).
|
||||
# restart_policy set to `any` so SIGTERM/clean-exit also triggers restart.
|
||||
#
|
||||
# NOTE: This is a TEST template — production deployments use a separate
|
||||
# parameterised template with stricter resource limits and secrets.
|
||||
|
||||
version: '3.9'
|
||||
|
||||
services:
|
||||
gateway:
|
||||
image: git.mosaicstack.dev/mosaicstack/stack/gateway@sha256:1069117740e00ccfeba357cae38c43f3729fe5ae702740ce474f6512414d7c02
|
||||
# Tag for human reference: sha-9f1a081 (post-#488 Dockerfile fix; smoke-tested locally)
|
||||
environment:
|
||||
# ── Tier ───────────────────────────────────────────────────────────────
|
||||
MOSAIC_TIER: federated
|
||||
|
||||
# ── Database ───────────────────────────────────────────────────────────
|
||||
DATABASE_URL: postgres://gateway:${POSTGRES_PASSWORD}@postgres:5432/mosaic
|
||||
|
||||
# ── Queue ──────────────────────────────────────────────────────────────
|
||||
VALKEY_URL: redis://valkey:6379
|
||||
|
||||
# ── Gateway ────────────────────────────────────────────────────────────
|
||||
GATEWAY_PORT: '3000'
|
||||
GATEWAY_CORS_ORIGIN: https://${HOST_FQDN}
|
||||
|
||||
# ── Auth ───────────────────────────────────────────────────────────────
|
||||
BETTER_AUTH_SECRET: ${BETTER_AUTH_SECRET}
|
||||
BETTER_AUTH_URL: https://${HOST_FQDN}
|
||||
|
||||
# ── Observability ──────────────────────────────────────────────────────
|
||||
OTEL_SERVICE_NAME: ${STACK_NAME:-mosaic-gateway}
|
||||
# OTEL_EXPORTER_OTLP_ENDPOINT: http://<collector>:4318
|
||||
|
||||
# ── AI Providers (uncomment to enable) ─────────────────────────────────
|
||||
# ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}
|
||||
# OPENAI_API_KEY: ${OPENAI_API_KEY}
|
||||
networks:
|
||||
- federated-test
|
||||
- traefik-public
|
||||
deploy:
|
||||
replicas: 1
|
||||
restart_policy:
|
||||
condition: any
|
||||
delay: 5s
|
||||
max_attempts: 3
|
||||
labels:
|
||||
- 'traefik.enable=true'
|
||||
- 'traefik.docker.network=traefik-public'
|
||||
- 'traefik.http.routers.${STACK_NAME}.rule=Host(`${HOST_FQDN}`)'
|
||||
- 'traefik.http.routers.${STACK_NAME}.entrypoints=websecure'
|
||||
- 'traefik.http.routers.${STACK_NAME}.tls=true'
|
||||
- 'traefik.http.routers.${STACK_NAME}.tls.certresolver=letsencrypt'
|
||||
- 'traefik.http.services.${STACK_NAME}.loadbalancer.server.port=3000'
|
||||
healthcheck:
|
||||
test:
|
||||
- 'CMD'
|
||||
- 'node'
|
||||
- '-e'
|
||||
- "require('http').get('http://127.0.0.1:3000/health',r=>process.exit(r.statusCode===200?0:1)).on('error',()=>process.exit(1))"
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 60s
|
||||
depends_on:
|
||||
- postgres
|
||||
- valkey
|
||||
|
||||
postgres:
|
||||
image: pgvector/pgvector:pg17
|
||||
environment:
|
||||
POSTGRES_USER: gateway
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||
POSTGRES_DB: mosaic
|
||||
volumes:
|
||||
- postgres-data:/var/lib/postgresql/data
|
||||
networks:
|
||||
- federated-test
|
||||
deploy:
|
||||
replicas: 1
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 5s
|
||||
max_attempts: 3
|
||||
healthcheck:
|
||||
test: ['CMD-SHELL', 'pg_isready -U gateway']
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 10s
|
||||
|
||||
valkey:
|
||||
image: valkey/valkey:8-alpine
|
||||
volumes:
|
||||
- valkey-data:/data
|
||||
networks:
|
||||
- federated-test
|
||||
deploy:
|
||||
replicas: 1
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 5s
|
||||
max_attempts: 3
|
||||
healthcheck:
|
||||
test: ['CMD', 'valkey-cli', 'ping']
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
start_period: 5s
|
||||
|
||||
volumes:
|
||||
postgres-data:
|
||||
valkey-data:
|
||||
|
||||
networks:
|
||||
federated-test:
|
||||
driver: overlay
|
||||
traefik-public:
|
||||
external: true
|
||||
@@ -0,0 +1,290 @@
|
||||
# Design — #791: Framework upgrades must not destroy operator-owned config under `~/.config/mosaic`
|
||||
|
||||
- **Issue:** mosaicstack/stack#791
|
||||
- **Branch:** `feat/791-upgrade-config-protection` (off `origin/main` `9745bc3f`)
|
||||
- **Author:** ms-791 worker lane
|
||||
- **Status:** Phase 1 — DESIGN, awaiting MS-LEAD confirmation before implementation
|
||||
- **Ratified scope (Mos-approved, not re-litigated):** deliver **(b) strict ownership separation [PRIMARY]** + **(a) transactional pre-update snapshot [safety net]** + **(d) regeneration-from-SSOT [recovery]**. **(c) periodic backup timer is DEFERRED** — noted as future work only.
|
||||
|
||||
---
|
||||
|
||||
## 1. Current updater behavior + exact wipe mechanism (evidence)
|
||||
|
||||
### 1.1 What runs on `mosaic update`
|
||||
|
||||
`mosaic update` re-seeds the framework by invoking the **bash installer** in sync-only, keep mode:
|
||||
|
||||
- `packages/mosaic/src/runtime/update-checker.ts:509` `buildReseedCommand()` returns
|
||||
`bash <frameworkRoot>/install.sh` with env `MOSAIC_SYNC_ONLY=1`, `MOSAIC_INSTALL_MODE=keep`,
|
||||
`MOSAIC_HOME=<mosaicHome>`.
|
||||
- The same `install.sh` is the direct/`tools/install.sh` upgrade path and the framework-vN migration path.
|
||||
|
||||
So the destructive surface is **`packages/mosaic/framework/install.sh`**.
|
||||
|
||||
### 1.2 The wipe
|
||||
|
||||
`sync_framework()` (`install.sh:177`) performs, in `keep` mode:
|
||||
|
||||
```
|
||||
rsync -a --delete --exclude .git --exclude .framework-version --exclude '*.pre-constitution.bak' \
|
||||
[--exclude "/$path" for each PRESERVE_PATHS entry] SOURCE_DIR/ TARGET_DIR/
|
||||
```
|
||||
|
||||
- `install.sh:199` — `rsync -a --delete`. **`--delete` prunes every path in `~/.config/mosaic`
|
||||
that is NOT present in the shipped framework source**, unless excluded.
|
||||
- `install.sh:47` — `PRESERVE_PATHS` is the **only** thing standing between `--delete` and operator
|
||||
data. It is a _denylist of exclusions_:
|
||||
```
|
||||
PRESERVE_PATHS=("CONSTITUTION.md" "AGENTS.md" "SOUL.md" "USER.md" "TOOLS.md" "STANDARDS.md"
|
||||
"memory" "sources" "credentials" "fleet/roster.yaml" "fleet/roster.json" "fleet/agents"
|
||||
"fleet/run" "fleet/backlog" "fleet/roles.local")
|
||||
```
|
||||
- The cp-fallback (no rsync) is equally destructive: `install.sh:223`
|
||||
`find "$TARGET_DIR" -mindepth 1 -maxdepth 1 ... -exec rm -rf {} +` then re-copies source, restoring
|
||||
only PRESERVE_PATHS globs.
|
||||
|
||||
**Root-cause model:** _"Everything under `~/.config/mosaic` is framework-owned and pruneable UNLESS
|
||||
explicitly preserved."_ Any operator path the list forgets is destroyed on the next upgrade.
|
||||
|
||||
### 1.3 The exact operator paths wiped
|
||||
|
||||
Cross-referencing the issue's operator-owned list against `PRESERVE_PATHS`:
|
||||
|
||||
| Operator path (issue #791) | In PRESERVE_PATHS? | Fate on `mosaic update` |
|
||||
| ----------------------------------------------------------------- | --------------------------------------- | ----------------------- |
|
||||
| `agents/*.conf` (per-agent runtime) | **NO** | **WIPED** |
|
||||
| `policy/*.md` (operator overlays) | **NO** | **WIPED** |
|
||||
| `*.local.md` (SOUL/USER/STANDARDS) | **NO** | **WIPED** |
|
||||
| harvester / SOP artifacts + timers | **NO** | **WIPED** |
|
||||
| `tools/_lib/credentials.json` | **NO** (`credentials/` dir ≠ this path) | **WIPED** |
|
||||
| `fleet/agents/*.env` | yes (`fleet/agents`, added by #631) | survives |
|
||||
| `memory/`, `fleet/roster.*`, `fleet/backlog`, `fleet/roles.local` | yes | survives |
|
||||
|
||||
The `fleet/agents`, `memory`, `fleet/backlog` entries were **retro-added after prior incidents**
|
||||
(#631). This whack-a-mole is the structural signature of a denylist.
|
||||
|
||||
**Stale-comment evidence:** `update-checker.ts:492` claims the reseed preserves
|
||||
"`SOUL/USER/*.local/credentials`" — but `PRESERVE_PATHS` contains **no `*.local` entry**. The code
|
||||
documents protection it does not deliver.
|
||||
|
||||
### 1.4 Second code path (TS) — already non-destructive, but drifted
|
||||
|
||||
`FileConfigAdapter.syncFramework()` (`packages/mosaic/src/config/file-adapter.ts:157`) →
|
||||
`syncDirectory()` (`packages/mosaic/src/platform/file-ops.ts:66`) is a **copy-overlay**: it copies
|
||||
source over target and skips preserved paths, but **never deletes** target paths absent from source
|
||||
(`file-ops.ts:77-109`). It is used by the wizard/init flow, not `mosaic update`.
|
||||
|
||||
Two problems remain:
|
||||
|
||||
1. Its `preservePaths` (`file-adapter.ts:164-185`) has **already diverged** from `install.sh` — it is
|
||||
**missing `fleet/backlog` and `fleet/roles.local`**. Two hand-maintained denylists, drifted. This
|
||||
is direct evidence for a single shared SSOT manifest.
|
||||
2. Even non-destructive, it will happily _overwrite_ an operator file that collides with a
|
||||
framework-shipped path unless that path is on its (incomplete) preserve list.
|
||||
|
||||
### 1.5 Existing snapshot is inadequate for rollback
|
||||
|
||||
`make_snapshot()`/`restore_snapshot()` (`install.sh:76-87`) copy `TARGET_DIR` to `mktemp -d` under
|
||||
`/tmp`, restore **only on `ERR/INT/TERM` trap**, and are **deleted on success** (`cleanup_snapshot`,
|
||||
`install.sh:345`). Consequences: ephemeral `/tmp`, no retention, no post-success rollback, and **no
|
||||
`mosaic restore`**. It is crash-safety only, not the transactional safety net #791 requires.
|
||||
|
||||
---
|
||||
|
||||
## 2. Fix (b) — Strict ownership separation [PRIMARY / root cause]
|
||||
|
||||
### 2.1 Ownership model (invert to allow-list)
|
||||
|
||||
Replace _"framework-owned unless preserved"_ with _"operator-owned unless framework-owned"_, resolved
|
||||
**per target path** with operator carve-outs winning inside shared framework subtrees.
|
||||
|
||||
Two declared lists, one SSOT data file shipped in the framework
|
||||
(`framework/framework-manifest.json`), consumed by **both** bash and TS:
|
||||
|
||||
- **`framework` globs** — paths the updater is entitled to create / overwrite / prune. Authored to
|
||||
match exactly what the framework ships in `packages/mosaic/framework/` (e.g. `CONSTITUTION.md`,
|
||||
`AGENTS.md`, `STANDARDS.md`, `TOOLS.md`, `guides/**`, `constitution/**`, `templates/**`, `tools/**`,
|
||||
`skills/**`, `mcp/**`, `defaults/**`, `fleet/examples/**`, `fleet/roles/**`, `fleet/profiles/**`,
|
||||
`fleet/roster.schema.json`).
|
||||
- **`operatorReserved` globs** — NEVER written or pruned, even nested inside a `framework` subtree;
|
||||
these **win** over `framework` (deny-wins / most-specific-wins). At minimum:
|
||||
`agents/**`, `policy/**`, `memory/**`, `sources/**`, `credentials/**`, `*.local.md`,
|
||||
`tools/_lib/credentials.json`, `fleet/roster.yaml`, `fleet/roster.json`, `fleet/agents/**`,
|
||||
`fleet/run/**`, `fleet/backlog/**`, `fleet/roles.local/**`, plus operator harvester/SOP artifacts.
|
||||
|
||||
### 2.2 Ownership resolution for a target path `P`
|
||||
|
||||
1. `P` matches `operatorReserved` → **operator-owned**: updater MUST NOT write, MUST NOT delete.
|
||||
2. else `P` matches `framework` → **framework-owned**: may overwrite; may prune **only if absent from
|
||||
the current SOURCE** (a genuinely retired framework file).
|
||||
3. else (matches neither) → **UNKNOWN ⇒ operator-owned by default (fail-safe)**: never delete.
|
||||
|
||||
Rule 3 is the actual root-cause fix: an operator path the manifest authors forget is still protected,
|
||||
because _unknown defaults to operator_. A denylist can never provide this guarantee.
|
||||
|
||||
### 2.3 Sync mechanism change (the mechanically-critical part)
|
||||
|
||||
`--delete` cannot express "prune only framework-owned" without re-enumerating every operator path
|
||||
(the denylist trap). So:
|
||||
|
||||
1. **Drop `--delete` from the bulk sync.** Copy `SOURCE → TARGET` non-destructively (writes/overwrites
|
||||
all framework files; deletes nothing). rsync without `--delete`, or the existing overlay copy.
|
||||
2. **Explicit manifest-scoped prune pass.** Iterate the **`framework` manifest** (not the whole tree);
|
||||
for each framework path present in `TARGET` but **absent in `SOURCE`**, delete it — after
|
||||
re-checking it does not match `operatorReserved`. Because the prune iterates only declared
|
||||
framework globs, operator/unknown paths are **structurally unreachable** by deletion.
|
||||
|
||||
This is implemented in both bash `sync_framework()` and TS `syncFramework()` from the shared manifest.
|
||||
A pure **prune-planner** function (TS) computes the delete-set from
|
||||
`(manifest, sourceListing, targetListing)` so the invariant is unit-testable in isolation.
|
||||
`PRESERVE_PATHS` becomes redundant (kept as a defense-in-depth alias mapping to `operatorReserved`, or
|
||||
removed) — either way the two lists stop drifting because they read one file.
|
||||
|
||||
### 2.4 HARD GATE test — "upgrade touches no path outside the manifest"
|
||||
|
||||
Filesystem-observation test in the existing `test-install-migration.sh` harness pattern (mktemp
|
||||
`MOSAIC_HOME`, `MOSAIC_SYNC_ONLY=1`), plus TS specs:
|
||||
|
||||
1. Seed a throwaway `TARGET` with a realistic operator mix — one sentinel per operator class:
|
||||
`agents/x.conf`, `policy/p.md`, `SOUL.local.md`, `memory/m.md`,
|
||||
`tools/_lib/credentials.json` (with a secret value), `fleet/agents/a.env`, `fleet/roster.yaml`,
|
||||
`harvester/sop.md`, **and a deliberately-unanticipated `unknown-operator-dir/x`**.
|
||||
2. Record hash+mtime of every sentinel.
|
||||
3. Run the upgrade from a `SOURCE` containing none of those operator paths.
|
||||
4. **Assert:** every sentinel exists, byte-identical, **mtime unchanged** (not even rewritten). The
|
||||
`unknown-operator-dir` surviving proves the fail-safe default — a denylist could not pass this case.
|
||||
5. **Positive controls:** framework files WERE updated; a retired framework file WAS pruned.
|
||||
6. **Property test** (TS prune-planner): for fuzzed operator paths, `deleteSet ⊆ {matches framework ∧
|
||||
in target ∧ not in source}` and `deleteSet ∩ operatorReserved = ∅`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Fix (a) — Transactional pre-update snapshot [safety net]
|
||||
|
||||
- **Destination:** `${XDG_STATE_HOME:-~/.local/state}/mosaic/backups/pre-update-<UTC-ts>/`.
|
||||
**Outside `~/.config/mosaic`** (so no future sync can sweep it) and outside any repo.
|
||||
- **Perms:** dir `0700`, files `0600` — enforced with `umask 077` around the copy **and** explicit
|
||||
`chmod`. Never world-readable.
|
||||
- **Scope:** the operator-owned surface (`operatorReserved` paths that exist) — bounded; does not copy
|
||||
the framework tree.
|
||||
- **Timing:** taken before ANY mutation in the upgrade flow.
|
||||
- **Post-sync verify + selective restore:** after sync, diff the operator surface against the snapshot;
|
||||
since (b) should never touch operator paths, any diff means a manifest bug — restore the affected
|
||||
paths from the snapshot and warn loudly. This is precisely (a) catching a miss in (b).
|
||||
- **Retention:** keep N most-recent (default 5; `MOSAIC_BACKUP_RETENTION` override); prune older.
|
||||
- **`mosaic restore`:** `--list` (default, dry-run) enumerates snapshots by timestamp;
|
||||
`--from <ts>` restores that snapshot over the operator surface, confirmation-gated. Reports
|
||||
counts/paths only.
|
||||
- **Secret-safety:** snapshot copy and restore never emit file **contents**; only paths/counts.
|
||||
Tests assert `0700/0600` and that no secret value appears in stdout/stderr.
|
||||
|
||||
---
|
||||
|
||||
## 4. Fix (d) — Regeneration-from-SSOT [recovery]
|
||||
|
||||
The incident's live blast radius: `fleet/agents/*.env` (systemd `EnvironmentFile` sources) gone →
|
||||
`mosaic-agent@<name>` boots **unit defaults** on restart (because `EnvironmentFile=-...` is
|
||||
absent-tolerant) → **silent identity/runtime/workdir downgrade**.
|
||||
|
||||
The SSOT for those `.env` files is the roster. The reconciler **already** separates a
|
||||
`regenerate-projections-from-roster` projection phase from lifecycle
|
||||
(`packages/mosaic/src/fleet/fleet-reconciler.ts:93,234`; env rendering in
|
||||
`generated-env-boundary.ts:149-264`).
|
||||
|
||||
**`mosaic fleet regen`** is therefore a **thin recovery-framed wrapper over the existing projection
|
||||
phase** — it does NOT reimplement fleet logic and does NOT preempt in-flight FCM cards (M4/M5):
|
||||
|
||||
- Regenerates derivable config (per-agent `*.env.generated`, unit files) from roster SSOT.
|
||||
- **Preview-first:** dry-run default; `--write` to apply. Idempotent.
|
||||
- **Never restarts agents** (the recovery order forbids restart-before-verify).
|
||||
- Prints the runbook's next step (verify `EnvironmentFile` resolves, THEN restart).
|
||||
|
||||
Alternatively documentable as `install.sh --relink` per the issue; `mosaic fleet regen` is preferred
|
||||
because it reuses the merged reconciler plumbing.
|
||||
|
||||
---
|
||||
|
||||
## 5. Secret-safety approach (secrev surface)
|
||||
|
||||
- Snapshots/backups: `0700`/`0600`, outside any repo, never world-readable. (§3)
|
||||
- No secret **value** ever emitted to logs/stdout/stderr by snapshot, restore, sync, or regen —
|
||||
paths/counts only. Adversarial test: a secret value placed in `tools/_lib/credentials.json` must
|
||||
never appear in installer or command output.
|
||||
- `tools/_lib/credentials.json` is an explicit `operatorReserved` carve-out inside the framework-owned
|
||||
`tools/**` subtree — it is never overwritten or pruned.
|
||||
- The HARD GATE test doubles as a secret-safety test (asserts the credentials sentinel is untouched).
|
||||
|
||||
---
|
||||
|
||||
## 6. Test plan (TDD, tests-first, ≥85% on new code, co-located `*.spec.ts`)
|
||||
|
||||
1. **Manifest SSOT parity** — bash and TS resolve identical framework/operator sets from the one file;
|
||||
a test fails if either path hard-codes a divergent list.
|
||||
2. **Manifest completeness** — every path shipped in `framework/` is covered by a `framework` glob (so
|
||||
a new shipped file cannot silently fall outside the manifest and become un-prunable/undeclared).
|
||||
3. **HARD GATE** — upgrade touches nothing outside the manifest, incl. the unanticipated-path case
|
||||
(§2.4).
|
||||
4. **Prune-planner** unit + property tests (§2.4.6).
|
||||
5. **Snapshot** — perms `0700/0600`, correct destination, retention prune, secret value absent from
|
||||
output.
|
||||
6. **Restore** — `--list` / `--from` round-trip restores operator surface byte-exact; confirmation
|
||||
gate; no secret leakage.
|
||||
7. **Regen** — roster→env projection deterministic + idempotent; dry-run makes no writes; `--write`
|
||||
restores `*.env`; **never** issues a lifecycle/restart call.
|
||||
8. **Cross-path regression** — TS `syncFramework` and bash `install.sh` agree on a shared fixture
|
||||
(closes the current #631-style drift).
|
||||
|
||||
Gates before every push: `pnpm typecheck && pnpm lint && pnpm format:check` + mosaic package tests
|
||||
green. Never `--no-verify`.
|
||||
|
||||
---
|
||||
|
||||
## 7. web1 recovery runbook (operator-agnostic; web1 specifics live in the issue as evidence only)
|
||||
|
||||
For a currently-wiped fleet EnvironmentFile state — **do NOT service-restart while
|
||||
`fleet/agents/*.env` is absent** (a restart boots unit defaults and silently downgrades identity):
|
||||
|
||||
1. **Regenerate:** `mosaic fleet regen --write` — rebuild `~/.config/mosaic/fleet/agents/*.env` from
|
||||
roster SSOT.
|
||||
2. **Verify each unit resolves to the intended runtime/workdir** _before_ any restart:
|
||||
`systemctl --user show mosaic-agent@<name> -p EnvironmentFile` and confirm the generated env exists
|
||||
and carries the intended `MOSAIC_AGENT_*` runtime/workdir values.
|
||||
3. **Only then** `systemctl --user restart mosaic-agent@<name>`, one unit at a time.
|
||||
|
||||
If config (not just fleet env) was lost, `mosaic restore --list` → `mosaic restore --from <ts>` before
|
||||
step 1.
|
||||
|
||||
---
|
||||
|
||||
## 8. Proposed PR split (reviewable; DAG-ordered)
|
||||
|
||||
| PR | Scope | Depends | Review focus |
|
||||
| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -------------------------- |
|
||||
| PR1 | **PRIMARY** — shared `framework-manifest.json` + ownership resolver + non-deleting sync + scoped prune (bash + TS) + **HARD GATE** + prune-planner tests | — | correctness (root fix) |
|
||||
| PR2 | **Safety net** — pre-update snapshot (`~/.local/state`, 0700/0600, retention) + post-sync verify/restore + `mosaic restore` | PR1 | **secrev** (backup/secret) |
|
||||
| PR3 | **Recovery** — `mosaic fleet regen` (projection-only, preview-first, no restart) + docs (upgrade-safety + recovery runbook) | PR1 | correctness + docs |
|
||||
|
||||
Rationale: PR1 closes the failure class on its own; if PR2/PR3 slip, the class stays fixed. Each PR is
|
||||
one reviewable unit with its own tests ≥85%. Independent review (author≠reviewer) on all; **secrev** on
|
||||
PR2 (and PR1's secret-sentinel assertions).
|
||||
|
||||
## 9. Deferred (noted per scope)
|
||||
|
||||
**(c) periodic backup timer** — a systemd user timer snapshotting operator dirs on a cadence
|
||||
(defense-in-depth for non-upgrade losses). Explicitly **out of scope now**; future phase.
|
||||
|
||||
## 10. Constraints honored
|
||||
|
||||
- **Framework-PR firewall:** manifest + logic are operator-agnostic; no SOUL/USER/operator specifics
|
||||
in framework code; web1 details are issue evidence only.
|
||||
- **Capacity-fill:** must not preempt M5-001 or #790; `fleet regen` reuses merged FCM-M3 plumbing and
|
||||
does not overlap FCM-M4/M5 migration cards.
|
||||
- **Delivery gates:** TDD tests-first, ≥85% new-code coverage, trunk-based squash PRs, independent
|
||||
review + secrev, completion = merged PR + descendant-main green + #791 closed.
|
||||
|
||||
---
|
||||
|
||||
**Requesting MS-LEAD confirmation of:** (1) the manifest allow-list + non-deleting-sync + scoped-prune
|
||||
approach as the (b) root-cause fix; (2) snapshot destination/retention + `mosaic restore` UX;
|
||||
(3) `mosaic fleet regen` as a projection-only wrapper; (4) the 3-PR split. Implementation begins only
|
||||
on your confirmation.
|
||||
@@ -0,0 +1,75 @@
|
||||
# Constitution Alpha — Definition-of-Done checklist + release notes
|
||||
|
||||
Drafted for the `v0.0.39-alpha` tag (Lead cuts after P5 #605 → P6 #607 → aiguide #8 merge).
|
||||
Maps every DoD §8 acceptance criterion to its merged evidence. Legend:
|
||||
**✅ merged on main** · **⏳ review-ready PR (pending merge)** · **🔲 Lead action**.
|
||||
|
||||
## DoD §8 green-checklist
|
||||
|
||||
| # | Acceptance criterion (DESIGN §8) | Status | Evidence / PR |
|
||||
| --- | ------------------------------------------------------------------------------------------------------ | ------ | ----------------- |
|
||||
| 1 | MIT `LICENSE` (root + framework) + `"license":"MIT"` in package.json | ✅ | P0 #570 |
|
||||
| 2 | Three credential-path sites + hook URL fast-failed (no private paths in `*.sh`/hooks) | ✅ | P0 #570 |
|
||||
| 3 | `verify-sanitized.sh` (two-class, `*.sh`+`*.md`, self-tested) wired **blocking** in CI | ✅ | P1 #572 |
|
||||
| 4 | Operator data purged from the full set (guides / tools / init-generator) | ✅ | P2 #572 |
|
||||
| 5 | `rails/`→`tools/` in **both** template families | ✅ | P2 #572 |
|
||||
| 6 | `jarvis-loop.json` deleted; `defaults/SOUL.md` → **neutral sanitized persona** (Q10 decision) | ✅ | P2 #572 |
|
||||
| 7 | `CONSTITUTION.md` extracted (gates one place, capability-verb, §1.4 split, no false "already loaded") | ✅ | P3 #575 / #577 |
|
||||
| 8 | `AGENTS.md`/`STANDARDS.md` out of `PRESERVE_PATHS` + seed-semantics → overwrite in **both** installers | ✅ | P4 #590 |
|
||||
| 9 | Snapshot + v2→v3 migration moving user edits to `.local`/`.bak`; `FRAMEWORK_VERSION=3` | ✅ | P4 #590 / #593 |
|
||||
| 10 | `mosaic-init --non-interactive` fail-closed persona | ✅ | P4 #590 |
|
||||
| 11 | **5-fixture migration matrix** green against **both** installers asserting **injected bytes** | ✅ | P4 #590 / #593 |
|
||||
| 12 | `compose-contract` built + composer unit test (per-tier anchor + Tier-3 byte-equality) | ⏳ | P5 #605 |
|
||||
| 13 | Resident line-count ceiling enforced (framework-owned resident files) | ⏳ | P6 #607 |
|
||||
| 14 | `CONTRIBUTING.md` + harness×gate compliance matrix | ⏳ | P6 #607 |
|
||||
| 15 | `aiguide` reconciled with the Constitution | ⏳ | aiguide #8 |
|
||||
| 16 | Each phase PR CI-green; alpha tag pushed + Gitea release published | 🔲 | Lead (post-merge) |
|
||||
|
||||
**Note on #6:** the DoD's literal "delete `defaults/SOUL.md`" was superseded by the resolved
|
||||
**Q10** decision — ship a _neutral, operator-agnostic_ example persona instead of deleting it. Main
|
||||
carries the sanitized 2.6 KB neutral SOUL.md ("Mosaic agent", no operator identity); the sanitization
|
||||
gate confirms it is PII-clean. Criterion met in spirit (no operator persona leaks) via the better option.
|
||||
|
||||
**Gate to flip 12–14 → ✅:** merge P5 #605 → P6 #607 (rebase auto-drops the dup format fix
|
||||
`adc7df2`/`9f6da92`) → aiguide #8, with `ci.yml` terminal-green on the merged head.
|
||||
|
||||
---
|
||||
|
||||
## Release notes — `v0.0.39-alpha` (Mosaic Framework Constitution, alpha)
|
||||
|
||||
### Mosaic Framework Constitution — Alpha
|
||||
|
||||
This release makes the Mosaic framework a **safe-to-open-source, fork-and-customize agent
|
||||
operating layer**. It separates the non-negotiable law from operator identity, makes
|
||||
customization survive upgrades, and wires the guarantees into CI.
|
||||
|
||||
**Highlights**
|
||||
|
||||
- **Constitution (L0).** The hard gates now live in one place — `CONSTITUTION.md` — authored in
|
||||
capability verbs, with a thin `AGENTS.md` dispatcher that references the law instead of restating
|
||||
it. Governance model in `constitution/LAYER-MODEL.md`.
|
||||
- **Public & sanitized.** MIT-licensed; all operator identity, private paths, and credential sites
|
||||
removed from shipped files. A self-tested `verify-sanitized.sh` gate (two rule classes) runs
|
||||
**blocking** in CI so re-contamination can't merge.
|
||||
- **Upgrade-safe customization.** Framework-owned files overwrite cleanly on upgrade while
|
||||
`SOUL.md`/`USER.md`/`*.local.md`/`credentials` are preserved. The v2→v3 migration snapshots first
|
||||
and moves any user-edited `AGENTS.md`/`STANDARDS.md` to `.pre-constitution.bak`/`.local.md` —
|
||||
never silently lost. Verified by a 5-fixture matrix across **both** installers.
|
||||
- **Operator overlays.** `mosaic compose-contract <harness>` merges your `*.local.md` deltas into
|
||||
the contract per harness, so customization reaches the model as one pre-merged blob.
|
||||
- **Cross-harness.** Single L0 source referenced (never restated) by Claude / Codex / OpenCode / Pi;
|
||||
tiered injection with a byte-equal Tier-3 fallback read.
|
||||
- **Guardrails in CI.** Resident line-count ceiling over framework-owned resident files; composer
|
||||
unit test; sanitization gate — all blocking.
|
||||
- **Docs.** `CONTRIBUTING.md` with the layer model, dual-installer parity rule, and a harness×gate
|
||||
**compliance matrix** (the Codex/OpenCode/Pi hook-parity gap is tracked for v2).
|
||||
|
||||
**Known limitations (accepted, documented in `CONTRIBUTING.md` §9)**
|
||||
|
||||
- Bare launches that bypass `mosaic` get base contracts only (no `*.local` overlays) and are not
|
||||
drift-checked by `mosaic doctor` — mitigated by the unconditional Tier-3 self-load + a nudge.
|
||||
- Codex/OpenCode/Pi mechanical hook parity, `policy/*.md` composition, and live-launch cross-harness
|
||||
verification are **v2**.
|
||||
|
||||
**Phase lineage:** P0 #570 · P1+P2 #572 · P3 #575/#577 · P4 #590/#593 · P5 #605 · P6 #607 ·
|
||||
aiguide #8 (umbrella #542).
|
||||
@@ -0,0 +1,63 @@
|
||||
# npm `@next` prerelease lane
|
||||
|
||||
Status: **IMPLEMENTED**
|
||||
|
||||
## Current behavior
|
||||
|
||||
`tools/install.sh --next` provides the prerelease integration lane for the permanent `next` branch.
|
||||
|
||||
The lane is fast-by-default:
|
||||
|
||||
1. Install framework files from the `next` source archive.
|
||||
2. Resolve the Gitea npm registry `next` dist-tag for the globally installed packages:
|
||||
|
||||
```bash
|
||||
npm view @mosaicstack/gateway@next version
|
||||
npm view @mosaicstack/mosaic@next version
|
||||
```
|
||||
|
||||
3. Require both resolved versions to share the same `next.<pipeline>` suffix, then install the exact resolved versions.
|
||||
4. If either `@next` package is missing, unreachable, mismatched, or fails to install, fall back to the source-build path at `next`.
|
||||
|
||||
`--next` never hard-fails solely because the prerelease npm dist-tag is unavailable.
|
||||
|
||||
## Published packages
|
||||
|
||||
The `next` publish pipeline publishes non-private `@mosaicstack/*` packages to the Mosaic Gitea npm registry:
|
||||
|
||||
```text
|
||||
https://git.mosaicstack.dev/api/packages/mosaicstack/npm/
|
||||
```
|
||||
|
||||
Observed `next` dist-tags after enabling the pipeline:
|
||||
|
||||
```text
|
||||
@mosaicstack/mosaic@next -> 0.0.49-next.1633
|
||||
@mosaicstack/gateway@next -> 0.0.7-next.1633
|
||||
```
|
||||
|
||||
The gateway also publishes a Docker image as `gateway:sha-<short>` on `next` merges. The installer fast path uses the npm gateway package when available; the Docker image is for deployed gateway/runtime harness flows.
|
||||
|
||||
## Explicit source lanes
|
||||
|
||||
Source builds remain available and are still the authority for explicit ref validation:
|
||||
|
||||
- `--dev` always builds from source.
|
||||
- `--ref <ref>` / `MOSAIC_REF=<ref>` wins over `--next` and uses the source path for that exact ref.
|
||||
|
||||
## Pipeline shape
|
||||
|
||||
1. Trigger on `next` merges.
|
||||
2. Compute the next prerelease version from the upcoming stable version plus the Woodpecker pipeline number (`<target-stable>-next.<CI_PIPELINE_NUMBER>`).
|
||||
3. Build and publish non-private packages in CI.
|
||||
4. Publish to the Mosaic Gitea npm registry with dist-tag `next`.
|
||||
5. Keep `latest` untouched; only main/release promotion can update `latest`.
|
||||
6. Publish gateway Docker images from `next` as `gateway:sha-<short>` only.
|
||||
|
||||
## Guardrails
|
||||
|
||||
- `@next` is mutable prerelease convenience, not a deployment pin.
|
||||
- Stable installs continue to use `@latest`.
|
||||
- Contributor validation remains available through `--dev --ref <branch>`.
|
||||
- Pipeline output traces every prerelease package back to the source commit on `next`.
|
||||
- The installer falls back to source rather than hard-failing on prerelease registry issues.
|
||||
@@ -0,0 +1,559 @@
|
||||
# Storage & Queue Abstraction — Middleware Architecture
|
||||
|
||||
Design
|
||||
Status: Design (retrofit required)
|
||||
date: 2026-04-02
|
||||
context: Agents coupled directly to infrastructure backends, bypassing intended middleware layer
|
||||
|
||||
---
|
||||
|
||||
## The Problem
|
||||
|
||||
Current packages are **direct adapters**, not **middleware**:
|
||||
| Package | Current State | Intended Design |
|
||||
|---------|---------------|-----------------|
|
||||
| `@mosaicstack/queue` | `ioredis` hardcoded | Interface → BullMQ OR local-files |
|
||||
| `@mosaicstack/db` | Drizzle + Postgres hardcoded | Interface → Postgres OR SQLite OR JSON/MD |
|
||||
| `@mosaicstack/memory` | pgvector required | Interface → pgvector OR sqlite-vec OR keyword-search |
|
||||
|
||||
## The gateway and TUI import these packages directly, which means they they're coupled to specific infrastructure. Users cannot run Mosaic Stack without Postgres + Valkey.
|
||||
|
||||
## The Intended Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Gateway / TUI / CLI │
|
||||
│ (agnostic of storage backend, talks to middleware) │
|
||||
└───────────────────────────┬─────────────────────────────────────┘
|
||||
│
|
||||
┌───────────────────┼───────────────────┐
|
||||
│ │ │
|
||||
▼─────────────────┴─────────────────┴─────────────────┘
|
||||
| | | |
|
||||
▼─────────────────┴───────────────────┴─────────────────┘
|
||||
| | | |
|
||||
Queue Storage Memory
|
||||
| | | |
|
||||
┌─────────┬─────────┬─────────┬─────────────────────────────────┐
|
||||
| BullMQ | | Local | | Postgres | SQLite | JSON/MD | pgvector | sqlite-vec | keyword |
|
||||
|(Valkey)| |(files) | | | | | |
|
||||
└─────────┴─────────┴─────────┴─────────────────────────────────┘
|
||||
```
|
||||
|
||||
The gateway imports the interface, not the backend. At startup it reads config and instantiates the correct adapter.
|
||||
|
||||
## The Drift
|
||||
|
||||
```typescript
|
||||
// What should have happened:
|
||||
gateway/queue.service.ts → @mosaicstack/queue (interface) → queue.adapter.ts
|
||||
|
||||
// What actually happened:
|
||||
gateway/queue.service.ts → @mosaicstack/queue → ioredis (hardcoded)
|
||||
```
|
||||
|
||||
## The Current State Analysis
|
||||
|
||||
### `@mosaicstack/queue` (packages/queue/src/queue.ts)
|
||||
|
||||
```typescript
|
||||
import Redis from 'ioredis'; // ← Direct import of backend
|
||||
|
||||
export function createQueue(config?: QueueConfig): QueueHandle {
|
||||
const url = config?.url ?? process.env['VALKEY_URL'] ?? DEFAULT_VALKEY_URL;
|
||||
const redis = new Redis(url, { maxRetriesPerRequest: 3 });
|
||||
// ...queue ops directly on redis...
|
||||
}
|
||||
```
|
||||
|
||||
**Problem:** `ioredis` is imported in the package, not the adapter interface. Consumers cannot swap backends.
|
||||
|
||||
### `@mosaicstack/db` (packages/db/src/client.ts)
|
||||
|
||||
> **Historical design specimen — status-only, not an operator instruction.** KBN-101 supersedes
|
||||
> this pre-split `DATABASE_URL` fallback shape; it cannot authorize runtime migration, DDL, or a
|
||||
> connection-string fallback. See the KBN-101 runner/role contract for the produced interface.
|
||||
|
||||
```typescript
|
||||
import { drizzle, type PostgresJsDatabase } from 'drizzle-orm/postgres-js';
|
||||
import postgres from 'postgres';
|
||||
|
||||
export function createDb(url?: string): DbHandle {
|
||||
const connectionString = url ?? process.env['DATABASE_URL'] ?? DEFAULT_DATABASE_URL;
|
||||
const sql = postgres(connectionString, { max: 20, idle_timeout: 30, connect_timeout: 5 });
|
||||
const db = drizzle(sql, { schema });
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
**Problem:** Drizzle + Postgres is hardcoded. No SQLite, JSON, or file-based options.
|
||||
|
||||
### `@mosaicstack/memory` (packages/memory/src/memory.ts)
|
||||
|
||||
```typescript
|
||||
import type { Db } from '@mosaicstack/db'; // ← Depends on Drizzle/PG
|
||||
|
||||
export function createMemory(db: Db): Memory {
|
||||
return {
|
||||
preferences: createPreferencesRepo(db),
|
||||
insights: createInsightsRepo(db),
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
**Problem:** Memory package is tightly coupled to `@mosaicstack/db` (which is Postgres-only). No alternative storage backends.
|
||||
|
||||
## The Target Interfaces
|
||||
|
||||
### Queue Interface
|
||||
|
||||
```typescript
|
||||
// packages/queue/src/types.ts
|
||||
export interface QueueAdapter {
|
||||
readonly name: string;
|
||||
|
||||
enqueue(queueName: string, payload: TaskPayload): Promise<void>;
|
||||
dequeue(queueName: string): Promise<TaskPayload | null>;
|
||||
length(queueName: string): Promise<number>;
|
||||
publish(channel: string, message: string): Promise<void>;
|
||||
subscribe(channel: string, handler: (message: string) => void): () => void;
|
||||
close(): Promise<void>;
|
||||
}
|
||||
|
||||
export interface TaskPayload {
|
||||
id: string;
|
||||
type: string;
|
||||
data: Record<string, unknown>;
|
||||
createdAt: string;
|
||||
}
|
||||
|
||||
export interface QueueConfig {
|
||||
type: 'bullmq' | 'local';
|
||||
url?: string; // For bullmq: Valkey/Redis URL
|
||||
dataDir?: string; // For local: directory for JSON persistence
|
||||
}
|
||||
```
|
||||
|
||||
### Storage Interface
|
||||
|
||||
```typescript
|
||||
// packages/storage/src/types.ts
|
||||
export interface StorageAdapter {
|
||||
readonly name: string;
|
||||
|
||||
// Entity CRUD
|
||||
create<T>(collection: string, data: O): Promise<T>;
|
||||
read<T>(collection: string, id: string): Promise<T | null>;
|
||||
update<T>(collection: string, id: string, data: Partial<O>): Promise<T | null>;
|
||||
delete(collection: string, id: string): Promise<boolean>;
|
||||
|
||||
// Queries
|
||||
find<T>(collection: string, filter: Record<string, unknown>): Promise<T[]>;
|
||||
findOne<T>(collection: string, filter: Record<string, unknown): Promise<T | null>;
|
||||
|
||||
// Bulk operations
|
||||
createMany<T>(collection: string, items: O[]): Promise<T[]>;
|
||||
updateMany<T>(collection: string, ids: string[], data: Partial<O>): Promise<number>;
|
||||
deleteMany(collection: string, ids: string[]): Promise<number>;
|
||||
|
||||
// Raw queries (for complex queries)
|
||||
query<T>(collection: string, query: string, params?: unknown[]): Promise<T[]>;
|
||||
|
||||
// Transaction support
|
||||
transaction<T>(fn: (tx: StorageTransaction) => Promise<T>): Promise<T>;
|
||||
|
||||
close(): Promise<void>;
|
||||
}
|
||||
|
||||
export interface StorageTransaction {
|
||||
commit(): Promise<void>;
|
||||
rollback(): Promise<void>;
|
||||
}
|
||||
|
||||
export interface StorageConfig {
|
||||
type: 'postgres' | 'sqlite' | 'files';
|
||||
url?: string; // For postgres
|
||||
path?: string; // For sqlite/files
|
||||
}
|
||||
```
|
||||
|
||||
### Memory Interface (Vector + Preferences)
|
||||
|
||||
```typescript
|
||||
// packages/memory/src/types.ts
|
||||
export interface MemoryAdapter {
|
||||
readonly name: string;
|
||||
|
||||
// Preferences (key-value storage)
|
||||
getPreference(userId: string, key: string): Promise<unknown | null>;
|
||||
setPreference(userId: string, key: string, value: unknown): Promise<void>;
|
||||
deletePreference(userId: string, key: string): Promise<boolean>;
|
||||
listPreferences(
|
||||
userId: string,
|
||||
category?: string,
|
||||
): Promise<Array<{ key: string; value: unknown }>>;
|
||||
|
||||
// Insights (with optional vector search)
|
||||
storeInsight(insight: NewInsight): Promise<Insight>;
|
||||
getInsight(id: string): Promise<Insight | null>;
|
||||
searchInsights(query: string, limit?: number, filter?: InsightFilter): Promise<SearchResult[]>;
|
||||
deleteInsight(id: string): Promise<boolean>;
|
||||
|
||||
// Embedding provider (optional, null = no vector search)
|
||||
readonly embedder?: EmbeddingProvider | null;
|
||||
|
||||
close(): Promise<void>;
|
||||
}
|
||||
|
||||
export interface NewInsight {
|
||||
id: string;
|
||||
userId: string;
|
||||
content: string;
|
||||
embedding?: number[]; // If embedder is available
|
||||
source: 'agent' | 'user' | 'summarization' | 'system';
|
||||
category: 'decision' | 'learning' | 'preference' | 'fact' | 'pattern' | 'general';
|
||||
relevanceScore: number;
|
||||
metadata?: Record<string, unknown>;
|
||||
createdAt: Date;
|
||||
decayedAt?: Date;
|
||||
}
|
||||
|
||||
export interface InsightFilter {
|
||||
userId?: string;
|
||||
category?: string;
|
||||
source?: string;
|
||||
minRelevance?: number;
|
||||
fromDate?: Date;
|
||||
toDate?: Date;
|
||||
}
|
||||
|
||||
export interface SearchResult {
|
||||
documentId: string;
|
||||
content: string;
|
||||
distance: number;
|
||||
metadata?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export interface MemoryConfig {
|
||||
type: 'pgvector' | 'sqlite-vec' | 'keyword';
|
||||
storage: StorageAdapter;
|
||||
embedder?: EmbeddingProvider;
|
||||
}
|
||||
|
||||
export interface EmbeddingProvider {
|
||||
embed(text: string): Promise<number[]>;
|
||||
embedBatch(texts: string[]): Promise<number[][]>;
|
||||
readonly dimensions: number;
|
||||
}
|
||||
```
|
||||
|
||||
## Three Tiers
|
||||
|
||||
### Tier 1: Local (Zero Dependencies)
|
||||
|
||||
**Target:** Single user, single machine, no external services
|
||||
|
||||
| Component | Backend | Storage |
|
||||
| --------- | --------------------------------------------- | ------------ |
|
||||
| Queue | In-process + JSON files in `~/.mosaic/queue/` |
|
||||
| Storage | SQLite (better-sqlite3) `~/.mosaic/data.db` |
|
||||
| Memory | Keyword search | SQLite table |
|
||||
| Vector | None | N/A |
|
||||
|
||||
**Dependencies:**
|
||||
|
||||
- `better-sqlite3` (bundled)
|
||||
- No Postgres, No Valkey, No pgvector
|
||||
|
||||
**Upgrade path:**
|
||||
|
||||
1. Run `mosaic gateway configure` → select "local" tier
|
||||
2. Gateway starts with SQLite database
|
||||
3. Optional: run `mosaic gateway upgrade --tier team` to migrate to Postgres
|
||||
|
||||
### Tier 2: Team (Postgres + Valkey)
|
||||
|
||||
**Target:** Multiple users, shared server, CI/CD environments
|
||||
|
||||
| Component | Backend | Storage |
|
||||
| --------- | -------------- | ------------------------------ |
|
||||
| Queue | BullMQ | Valkey |
|
||||
| Storage | Postgres | Shared PG instance |
|
||||
| Memory | pgvector | Postgres with vector extension |
|
||||
| Vector | LLM embeddings | Configured provider |
|
||||
|
||||
**Dependencies:**
|
||||
|
||||
- PostgreSQL 17+ with pgvector extension
|
||||
- Valkey (Redis-compatible)
|
||||
- LLM provider for embeddings
|
||||
|
||||
**Migration from Local → Team:**
|
||||
|
||||
1. `mosaic gateway backup` → creates dump of SQLite database
|
||||
2. `mosaic gateway upgrade --tier team` → restores to Postgres
|
||||
3. Queue replays from BullMQ (may need manual reconciliation for in-flight jobs)
|
||||
4. Memory embeddings regenerated if vector search was new
|
||||
|
||||
### Tier 3: Enterprise (Clustered)
|
||||
|
||||
**Target:** Large teams, multi-region, high availability
|
||||
|
||||
| Component | Backend | Storage |
|
||||
| --------- | --------------------------- | ----------------------------- |
|
||||
| Queue | BullMQ cluster | Multiple Valkey nodes |
|
||||
| Storage | Postgres cluster | Primary + replicas |
|
||||
| Memory | Dedicated vector DB | Qdrant, Pinecone, or pgvector |
|
||||
| Vector | Dedicated embedding service | Separate microservice |
|
||||
|
||||
## MarkdownDB Integration
|
||||
|
||||
For file-based storage, we use [MarkdownDB](https://markdowndb.com) to parse MD files into queryable data.
|
||||
|
||||
**What it provides:**
|
||||
|
||||
- Parses frontmatter (YAML/JSON/TOML)
|
||||
- Extracts links, tags, metadata
|
||||
- Builds index in JSON or SQLite
|
||||
- Queryable via SQL-like interface
|
||||
|
||||
**Usage in Mosaic:**
|
||||
|
||||
```typescript
|
||||
// Local tier with MD files for documents
|
||||
const storage = createStorageAdapter({
|
||||
type: 'files',
|
||||
path: path.join(mosaicHome, 'docs'),
|
||||
markdowndb: {
|
||||
parseFrontmatter: true,
|
||||
extractLinks: true,
|
||||
indexFile: 'index.json',
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Dream Mode — Memory Consolidation
|
||||
|
||||
Automated equivalent to Claude Code's "Dream: Memory Consolidation" cycle
|
||||
|
||||
**Trigger:** Every 24 hours (if 5+ sessions active)
|
||||
|
||||
**Phases:**
|
||||
|
||||
1. **Orient** — What happened, what's the current state
|
||||
- Scan recent session logs
|
||||
- Identify active tasks, missions, conversations
|
||||
- Calculate time window (last 24h)
|
||||
|
||||
2. **Gather** — Pull in relevant context
|
||||
- Load conversations, decisions, agent logs
|
||||
- Extract key interactions and outcomes
|
||||
- Identify patterns and learnings
|
||||
|
||||
3. **Consolidate** — Summarize and compress
|
||||
- Generate summary of the last 24h
|
||||
- Extract key decisions and their rationale
|
||||
- Identify recurring patterns
|
||||
- Compress verbose logs into concise insights
|
||||
|
||||
4. **Prune** — Archive and cleanup
|
||||
- Archive raw session files to dated folders
|
||||
- Delete redundant/temporary data
|
||||
- Update MEMORY.md with consolidated content
|
||||
- Update insight relevance scores
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```typescript
|
||||
// In @mosaicstack/dream (new package)
|
||||
export async function runDreamCycle(config: DreamConfig): Promise<DreamResult> {
|
||||
const memory = await loadMemoryAdapter(config.storage);
|
||||
|
||||
// Orient
|
||||
const sessions = await memory.getRecentSessions(24 * 60 * 60 * 1000);
|
||||
if (sessions.length < 5) return { skipped: true, reason: 'insufficient_sessions' };
|
||||
|
||||
// Gather
|
||||
const context = await gatherContext(memory, sessions);
|
||||
|
||||
// Consolidate
|
||||
const consolidated = await consolidateWithLLM(context, config.llm);
|
||||
|
||||
// Prune
|
||||
await pruneArchivedData(memory, config.retention);
|
||||
|
||||
// Store consolidated insights
|
||||
await memory.storeInsights(consolidated.insights);
|
||||
|
||||
return {
|
||||
sessionsProcessed: sessions.length,
|
||||
insightsCreated: consolidated.insights.length,
|
||||
bytesPruned: consolidated.bytesRemoved,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Retrofit Plan
|
||||
|
||||
### Phase 1: Interface Extraction (2-3 days)
|
||||
|
||||
**Goal:** Define interfaces without changing existing behavior
|
||||
|
||||
1. Create `packages/queue/src/types.ts` with `QueueAdapter` interface
|
||||
2. Create `packages/storage/src/types.ts` with `StorageAdapter` interface
|
||||
3. Create `packages/memory/src/types.ts` with `MemoryAdapter` interface (refactor existing)
|
||||
4. Add adapter registry pattern to each package
|
||||
5. No breaking changes — existing code continues to work
|
||||
|
||||
### Phase 2: Refactor Existing to Adapters (3-5 days)
|
||||
|
||||
**Goal:** Move existing implementations behind adapters
|
||||
|
||||
#### 2.1 Queue Refactor
|
||||
|
||||
1. Rename `packages/queue/src/queue.ts` → `packages/queue/src/adapters/bullmq.ts`
|
||||
2. Create `packages/queue/src/index.ts` to export factory function
|
||||
3. Factory function reads config, instantiates correct adapter
|
||||
4. Update gateway imports to use factory
|
||||
|
||||
#### 2.2 Storage Refactor
|
||||
|
||||
1. Create `packages/storage/` (new package)
|
||||
2. Move Drizzle logic to `packages/storage/src/adapters/postgres.ts`
|
||||
3. Create SQLite adapter in `packages/storage/src/adapters/sqlite.ts`
|
||||
4. Update gateway to use storage factory
|
||||
5. Deprecate direct `@mosaicstack/db` imports
|
||||
|
||||
#### 2.3 Memory Refactor
|
||||
|
||||
1. Extract existing logic to `packages/memory/src/adapters/pgvector.ts`
|
||||
2. Create keyword adapter in `packages/memory/src/adapters/keyword.ts`
|
||||
3. Update vector-store.ts to be adapter-agnostic
|
||||
|
||||
### Phase 3: Local Tier Implementation (2-3 days)
|
||||
|
||||
**Goal:** Zero-dependency baseline
|
||||
|
||||
1. Implement `packages/queue/src/adapters/local.ts` (in-process + JSON persistence)
|
||||
2. Implement `packages/storage/src/adapters/files.ts` (JSON + MD via MarkdownDB)
|
||||
3. Implement `packages/memory/src/adapters/keyword.ts` (TF-IDF search)
|
||||
4. Add `packages/dream/` for consolidation cycle
|
||||
5. Wire up local tier in gateway startup
|
||||
|
||||
### Phase 4: Configuration System (1-2 days)
|
||||
|
||||
**Goal:** Runtime backend selection
|
||||
|
||||
1. Create `packages/config/src/storage.ts` for storage configuration
|
||||
2. Add `mosaic.config.ts` schema with storage tier settings
|
||||
3. Update gateway to read config on startup
|
||||
4. Add `mosaic gateway configure` CLI command
|
||||
5. Add tier migration commands (`mosaic gateway upgrade`)
|
||||
|
||||
### Phase 5: Testing & Documentation (2-3 days)
|
||||
|
||||
1. Unit tests for each adapter
|
||||
2. Integration tests for factory pattern
|
||||
3. Migration tests (local → team)
|
||||
4. Update README and architecture docs
|
||||
5. Add configuration guide
|
||||
|
||||
---
|
||||
|
||||
## File Changes Summary
|
||||
|
||||
### New Files
|
||||
|
||||
```
|
||||
packages/
|
||||
├── config/
|
||||
│ └── src/
|
||||
│ ├── storage.ts # Storage config schema
|
||||
│ └── index.ts
|
||||
├── dream/ # NEW: Dream mode consolidation
|
||||
│ ├── src/
|
||||
│ │ ├── index.ts
|
||||
│ │ ├── orient.ts
|
||||
│ │ ├── gather.ts
|
||||
│ │ ├── consolidate.ts
|
||||
│ │ └── prune.ts
|
||||
│ └── package.json
|
||||
├── queue/
|
||||
│ └── src/
|
||||
│ ├── types.ts # NEW: QueueAdapter interface
|
||||
│ ├── index.ts # NEW: Factory function
|
||||
│ └── adapters/
|
||||
│ ├── bullmq.ts # MOVED from queue.ts
|
||||
│ └── local.ts # NEW: In-process adapter
|
||||
├── storage/ # NEW: Storage abstraction
|
||||
│ ├── src/
|
||||
│ │ ├── types.ts # StorageAdapter interface
|
||||
│ │ ├── index.ts # Factory function
|
||||
│ │ └── adapters/
|
||||
│ │ ├── postgres.ts # MOVED from @mosaicstack/db
|
||||
│ │ ├── sqlite.ts # NEW: SQLite adapter
|
||||
│ │ └── files.ts # NEW: JSON/MD adapter
|
||||
│ └── package.json
|
||||
└── memory/
|
||||
└── src/
|
||||
├── types.ts # UPDATED: MemoryAdapter interface
|
||||
├── index.ts # UPDATED: Factory function
|
||||
└── adapters/
|
||||
├── pgvector.ts # EXTRACTED from existing code
|
||||
├── sqlite-vec.ts # NEW: SQLite with vectors
|
||||
└── keyword.ts # NEW: TF-IDF search
|
||||
```
|
||||
|
||||
### Modified Files
|
||||
|
||||
```
|
||||
packages/
|
||||
├── db/ # DEPRECATED: Logic moved to storage adapters
|
||||
├── queue/
|
||||
│ └── src/
|
||||
│ └── queue.ts # → adapters/bullmq.ts
|
||||
├── memory/
|
||||
│ ├── src/
|
||||
│ │ ├── memory.ts # → use factory
|
||||
│ │ ├── insights.ts # → use factory
|
||||
│ │ └── preferences.ts # → use factory
|
||||
│ └── package.json # Remove pgvector from dependencies
|
||||
└── gateway/
|
||||
└── src/
|
||||
├── database/
|
||||
│ └── database.module.ts # Update to use storage factory
|
||||
├── memory/
|
||||
│ └── memory.module.ts # Update to use memory factory
|
||||
└── queue/
|
||||
└── queue.module.ts # Update to use queue factory
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Breaking Changes
|
||||
|
||||
1. **`@mosaicstack/db`** → **`@mosaicstack/storage`** (with migration guide)
|
||||
2. Direct `ioredis` imports → Use `@mosaicstack/queue` factory
|
||||
3. Direct `pgvector` queries → Use `@mosaicstack/memory` factory
|
||||
4. Gateway startup now requires storage config (defaults to local)
|
||||
|
||||
## Non-Breaking Migration Path
|
||||
|
||||
1. Existing deployments with Postgres/Valkey continue to work (default config)
|
||||
2. New deployments can choose local tier
|
||||
3. Migration commands available when ready to upgrade
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Local tier runs with zero external dependencies
|
||||
- [ ] All three tiers (local, team, enterprise) work correctly
|
||||
- [ ] Factory pattern correctly selects backend at runtime
|
||||
- [ ] Migration from local → team preserves all data
|
||||
- [ ] Dream mode consolidates 24h of sessions
|
||||
- [ ] Documentation covers all three tiers and migration paths
|
||||
- [ ] All existing tests pass
|
||||
- [ ] New adapters have >80% coverage
|
||||
@@ -0,0 +1,106 @@
|
||||
# Mosaic Federation — Admin CLI Reference
|
||||
|
||||
Available since: FED-M2
|
||||
|
||||
## Grant Management
|
||||
|
||||
### Create a grant
|
||||
|
||||
```bash
|
||||
mosaic federation grant create --user <userId> --peer <peerId> --scope <scope-file.json>
|
||||
```
|
||||
|
||||
The scope file defines what resources and rows the peer may access:
|
||||
|
||||
```json
|
||||
{
|
||||
"resources": ["tasks", "notes"],
|
||||
"excluded_resources": ["credentials"],
|
||||
"max_rows_per_query": 100
|
||||
}
|
||||
```
|
||||
|
||||
Valid resource values: `tasks`, `notes`, `credentials`, `teams`, `users`
|
||||
|
||||
### List grants
|
||||
|
||||
```bash
|
||||
mosaic federation grant list [--peer <peerId>] [--status pending|active|revoked|expired]
|
||||
```
|
||||
|
||||
Shows all federation grants, optionally filtered by peer or status.
|
||||
|
||||
### Show a grant
|
||||
|
||||
```bash
|
||||
mosaic federation grant show <grantId>
|
||||
```
|
||||
|
||||
Display details of a single grant, including its scope, activation timestamp, and status.
|
||||
|
||||
### Revoke a grant
|
||||
|
||||
```bash
|
||||
mosaic federation grant revoke <grantId> [--reason "Reason text"]
|
||||
```
|
||||
|
||||
Revoke an active grant immediately. Revoked grants cannot be reactivated. The optional reason is stored in the audit log.
|
||||
|
||||
### Generate enrollment token
|
||||
|
||||
```bash
|
||||
mosaic federation grant token <grantId> [--ttl <seconds>]
|
||||
```
|
||||
|
||||
Generate a single-use enrollment token for the grant. The default TTL is 900 seconds (15 minutes); maximum 15 minutes.
|
||||
|
||||
Output includes the token and the full enrollment URL for the peer to use.
|
||||
|
||||
## Peer Management
|
||||
|
||||
### Add a peer (remote enrollment)
|
||||
|
||||
```bash
|
||||
mosaic federation peer add <enrollment-url>
|
||||
```
|
||||
|
||||
Enroll a remote peer using the enrollment URL obtained from a grant token. The command:
|
||||
|
||||
1. Generates a P-256 ECDSA keypair locally
|
||||
2. Creates a certificate signing request (CSR)
|
||||
3. Submits the CSR to the enrollment URL
|
||||
4. Verifies the returned certificate includes the correct custom OIDs (grant ID and subject user ID)
|
||||
5. Seals the private key at rest using `BETTER_AUTH_SECRET`
|
||||
6. Stores the peer record and sealed key in the local gateway database
|
||||
|
||||
Once enrollment completes, the peer can authenticate using the certificate and private key.
|
||||
|
||||
### List peers
|
||||
|
||||
```bash
|
||||
mosaic federation peer list
|
||||
```
|
||||
|
||||
Shows all enrolled peers, including their certificate fingerprints and activation status.
|
||||
|
||||
## REST API Reference
|
||||
|
||||
All CLI commands call the local gateway admin API. Equivalent REST endpoints:
|
||||
|
||||
| CLI Command | REST Endpoint | Method |
|
||||
| ------------ | ------------------------------------------------------------------------------------------- | ----------------- |
|
||||
| grant create | `/api/admin/federation/grants` | POST |
|
||||
| grant list | `/api/admin/federation/grants` | GET |
|
||||
| grant show | `/api/admin/federation/grants/:id` | GET |
|
||||
| grant revoke | `/api/admin/federation/grants/:id/revoke` | PATCH |
|
||||
| grant token | `/api/admin/federation/grants/:id/tokens` | POST |
|
||||
| peer list | `/api/admin/federation/peers` | GET |
|
||||
| peer add | `/api/admin/federation/peers/keypair` + enrollment + `/api/admin/federation/peers/:id/cert` | POST, POST, PATCH |
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **Enrollment tokens** are single-use and expire in 15 minutes (not configurable beyond 15 minutes)
|
||||
- **Peer private keys** are encrypted at rest using AES-256-GCM, keyed from `BETTER_AUTH_SECRET`
|
||||
- **Custom OIDs** in issued certificates are verified post-issuance: the grant ID and subject user ID must match the certificate extensions
|
||||
- **Grant activation** is atomic — concurrent enrollment attempts for the same grant are rejected
|
||||
- **Revoked grants** cannot be activated; peers attempting to use a revoked grant's token will be rejected
|
||||
@@ -0,0 +1,368 @@
|
||||
# Mosaic Stack — Federation Implementation Milestones
|
||||
|
||||
**Companion to:** `PRD.md`
|
||||
**Approach:** Each milestone is a verifiable slice. A milestone is "done" only when its acceptance tests pass in CI against a real (not mocked) dependency stack.
|
||||
|
||||
---
|
||||
|
||||
## Milestone Dependency Graph
|
||||
|
||||
```
|
||||
M1 (federated tier infra)
|
||||
└── M2 (Step-CA + grant schema + CLI)
|
||||
└── M3 (mTLS handshake + list/get + scope enforcement)
|
||||
├── M4 (search + audit + rate limit)
|
||||
│ └── M5 (cache + offline degradation + OTEL)
|
||||
├── M6 (revocation + auto-renewal) ◄── can start after M3
|
||||
└── M7 (multi-user hardening + e2e suite) ◄── depends on M4+M5+M6
|
||||
```
|
||||
|
||||
M5 and M6 can run in parallel once M4 is merged.
|
||||
|
||||
---
|
||||
|
||||
## Test Strategy (applies to all milestones)
|
||||
|
||||
Three layers, all required before a milestone ships:
|
||||
|
||||
| Layer | Scope | Runtime |
|
||||
| ------------------ | --------------------------------------------- | ------------------------------------------------------------------------ |
|
||||
| **Unit** | Per-module logic, pure functions, adapters | Vitest, no I/O |
|
||||
| **Integration** | Single gateway against real PG/Valkey/Step-CA | Vitest + Docker Compose test profile |
|
||||
| **Federation E2E** | Two gateways on a Docker network, real mTLS | Playwright/custom harness (`tools/federation-harness/`) introduced in M3 |
|
||||
|
||||
Every milestone adds tests to these layers. A milestone cannot be claimed complete if the federation E2E harness fails (applies from M3 onward).
|
||||
|
||||
**Quality gates per milestone** (same as stack-wide):
|
||||
|
||||
- `pnpm typecheck` green
|
||||
- `pnpm lint` green
|
||||
- `pnpm test` green (unit + integration)
|
||||
- `pnpm test:federation` green (M3+)
|
||||
- Independent code review passed
|
||||
- Docs updated (`docs/federation/`)
|
||||
- Merged PR on `main`, CI terminal green, linked issue closed
|
||||
|
||||
---
|
||||
|
||||
## M1 — Federated Tier Infrastructure
|
||||
|
||||
**Goal:** A gateway can run in `federated` tier with containerized Postgres + Valkey + pgvector, with no federation logic active yet.
|
||||
|
||||
**Scope:**
|
||||
|
||||
- Add `"tier": "federated"` to `mosaic.config.json` schema and validators
|
||||
- Docker Compose `federated` profile (`docker-compose.federated.yml`) adds: Postgres+pgvector (5433), Valkey (6380), dedicated volumes
|
||||
- Tier detector in gateway bootstrap: reads config, asserts required services reachable, refuses to start otherwise
|
||||
- **Historical/status only:** the prior startup-provisioning statement is superseded. Runtime/startup extension provisioning is forbidden. PostgreSQL activation remains non-operative with no current command authority until KBN-101-00, KBN-101-03, and KBN-101-05 land; this record authorizes no current DDL, Compose/init, or startup path.
|
||||
- Migration logic: safe upgrade path from `local`/`standalone` → `federated` (data export/import script, one-way)
|
||||
- `mosaic doctor` reports tier + service health
|
||||
- Gateway continues to serve as a normal standalone instance (no federation yet)
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
- `mosaic.config.json` schema v2 (tier enum includes `federated`)
|
||||
- `apps/gateway/src/bootstrap/tier-detector.ts`
|
||||
- `docker-compose.federated.yml`
|
||||
- `scripts/migrate-to-federated.ts`
|
||||
- Updated `mosaic doctor` output
|
||||
- Updated `packages/storage/src/adapters/postgres.ts` with pgvector support
|
||||
|
||||
**Acceptance tests:**
|
||||
| # | Test | Layer |
|
||||
| - | ---------------------------------------------------------------------------------------- | ----------- |
|
||||
| 1 | Gateway boots in `federated` tier with all services present | Integration |
|
||||
| 2 | Gateway refuses to boot in `federated` tier when Postgres unreachable (fail-fast, clear) | Integration |
|
||||
| 3 | `pgvector` extension available in target DB (`SELECT * FROM pg_extension WHERE extname='vector'`) | Integration |
|
||||
| 4 | Migration script moves a populated `local` (PGlite) instance to `federated` (Postgres) with no data loss | Integration |
|
||||
| 5 | `mosaic doctor` reports correct tier and all services green | Unit |
|
||||
| 6 | Existing standalone behavior regression: agent session works end-to-end, no federation references | E2E (single-gateway) |
|
||||
|
||||
**Estimated budget:** ~20K tokens (infra + config + migration script)
|
||||
**Risk notes:** Pgvector install on existing PG installs is occasionally finicky; test the migration path on a realistic DB snapshot.
|
||||
|
||||
---
|
||||
|
||||
## M2 — Step-CA + Grant Schema + Admin CLI
|
||||
|
||||
**Goal:** An admin can create a federation grant and its counterparty can enroll. No runtime traffic flows yet.
|
||||
|
||||
**Scope:**
|
||||
|
||||
- Embed Step-CA as a Docker Compose sidecar with a persistent CA volume
|
||||
- Gateway exposes a short-lived enrollment endpoint (single-use token from the grant)
|
||||
- DB schema: `federation_grants`, `federation_peers`, `federation_audit_log` (table only, not yet written to)
|
||||
- Sealed storage for `client_key_pem` using the existing credential sealing key
|
||||
- Admin CLI:
|
||||
- `mosaic federation grant create --user <id> --peer <host> --scope <file>`
|
||||
- `mosaic federation grant list`
|
||||
- `mosaic federation grant show <id>`
|
||||
- `mosaic federation peer add <enrollment-url>`
|
||||
- `mosaic federation peer list`
|
||||
- Step-CA signs the cert with SAN OIDs for `grantId` + `subjectUserId`
|
||||
- Grant status transitions: `pending` → `active` on successful enrollment
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
- `packages/db` migration: three federation tables + enum types
|
||||
- `apps/gateway/src/federation/ca.service.ts` (Step-CA client)
|
||||
- `apps/gateway/src/federation/grants.service.ts`
|
||||
- `apps/gateway/src/federation/enrollment.controller.ts`
|
||||
- `packages/mosaic/src/commands/federation/` (grant + peer subcommands)
|
||||
- `docker-compose.federated.yml` adds Step-CA service
|
||||
- Scope JSON schema + validator
|
||||
|
||||
**Acceptance tests:**
|
||||
| # | Test | Layer |
|
||||
| - | ---------------------------------------------------------------------------------------- | ----------- |
|
||||
| 1 | `grant create` writes a `pending` row with a scoped bundle | Integration |
|
||||
| 2 | Enrollment endpoint signs a CSR and returns a cert with expected SAN OIDs | Integration |
|
||||
| 3 | Enrollment token is single-use; second attempt returns 410 | Integration |
|
||||
| 4 | Cert `subjectUserId` OID matches the grant's `subject_user_id` | Unit |
|
||||
| 5 | `client_key_pem` is at-rest encrypted; raw DB read shows ciphertext, not PEM | Integration |
|
||||
| 6 | `peer add <url>` on Server A yields an `active` peer record with a valid cert + key | E2E (two gateways, no traffic) |
|
||||
| 7 | Scope JSON with unknown resource type rejected at `grant create` | Unit |
|
||||
| 8 | `grant list` and `peer list` render active / pending / revoked accurately | Unit |
|
||||
|
||||
**Estimated budget:** ~30K tokens (schema + CA integration + CLI + sealing)
|
||||
**Risk notes:** Step-CA's API surface is well-documented but the sealing integration with existing provider-credential encryption is a cross-module concern — walk that seam deliberately.
|
||||
|
||||
---
|
||||
|
||||
## M3 — mTLS Handshake + `list` + `get` with Scope Enforcement
|
||||
|
||||
**Goal:** Two federated gateways exchange real data over mTLS with scope intersecting native RBAC.
|
||||
|
||||
**Scope:**
|
||||
|
||||
- `FederationClient` (outbound): picks cert from `federation_peers`, does mTLS call
|
||||
- `FederationServer` (inbound): NestJS guard validates client cert, extracts `grantId` + `subjectUserId`, loads grant
|
||||
- Scope enforcement pipeline:
|
||||
1. Resource allowlist / excluded-list check
|
||||
2. Native RBAC evaluation as the `subjectUserId`
|
||||
3. Scope filter intersection (`include_teams`, `include_personal`)
|
||||
4. `max_rows_per_query` cap
|
||||
- Verbs: `list`, `get`, `capabilities`
|
||||
- Gateway query layer accepts `source: "local" | "federated:<host>" | "all"`; fan-out for `"all"`
|
||||
- **Federation E2E harness** (`tools/federation-harness/`): docker-compose.two-gateways.yml, seed script, assertion helpers — this is its own deliverable
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
- `apps/gateway/src/federation/client/federation-client.service.ts`
|
||||
- `apps/gateway/src/federation/server/federation-auth.guard.ts`
|
||||
- `apps/gateway/src/federation/server/scope.service.ts`
|
||||
- `apps/gateway/src/federation/server/verbs/{list,get,capabilities}.controller.ts`
|
||||
- `apps/gateway/src/federation/client/query-source.service.ts` (fan-out/merge)
|
||||
- `tools/federation-harness/` (compose + seed + test helpers)
|
||||
- `packages/types` — federation request/response DTOs in `federation.dto.ts`
|
||||
|
||||
**Acceptance tests:**
|
||||
| # | Test | Layer |
|
||||
| -- | -------------------------------------------------------------------------------------------------------- | ----- |
|
||||
| 1 | A→B `list tasks` returns subjectUser's tasks intersected with scope | E2E |
|
||||
| 2 | A→B `list tasks` with `include_teams: [T1]` excludes T2 tasks the user owns | E2E |
|
||||
| 3 | A→B `get credential <id>` returns 403 when `credentials` is in `excluded_resources` | E2E |
|
||||
| 4 | Client presenting cert for grant X cannot query subjectUser of grant Y (cross-user isolation) | E2E |
|
||||
| 5 | Cert signed by untrusted CA rejected at TLS layer (no NestJS handler reached) | E2E |
|
||||
| 6 | Malformed SAN OIDs → 401; cert valid but grant revoked in DB → 403 | Integration |
|
||||
| 7 | `max_rows_per_query` caps response; request for more paginated | Integration |
|
||||
| 8 | `source: "all"` fan-out merges local + federated results, each tagged with `_source` | Integration |
|
||||
| 9 | Federation responses never persist: verify DB row count unchanged after `list` round-trip | E2E |
|
||||
| 10 | Scope cannot grant more than native RBAC: user without access to team T still gets [] even if scope allows T | E2E |
|
||||
|
||||
**Estimated budget:** ~40K tokens (largest milestone — core federation logic + harness)
|
||||
**Risk notes:** This is the critical trust boundary. Code review should focus on scope enforcement bypass and cert-SAN-spoofing paths. Every 403/401 path needs a test.
|
||||
|
||||
---
|
||||
|
||||
## M4 — `search` Verb + Audit Log + Rate Limit
|
||||
|
||||
**Goal:** Keyword search over allowed resources with full audit and per-grant rate limiting.
|
||||
|
||||
**Scope:**
|
||||
|
||||
- `search` verb across `resources` allowlist (intersection of scope + native RBAC)
|
||||
- Keyword search (reuse existing `packages/memory/src/adapters/keyword.ts`); pgvector search stays out of v1 search verb
|
||||
- Every federated request (all verbs) writes to `federation_audit_log`: `grant_id`, `verb`, `resource`, `query_hash`, `outcome`, `bytes_out`, `latency_ms`
|
||||
- No request body captured; `query_hash` is SHA-256 of normalized query params
|
||||
- Token-bucket rate limit per grant (default 60/min, override per grant)
|
||||
- 429 response with `Retry-After` header and structured body
|
||||
- 90-day hot retention for audit log; cold-tier rollover deferred to M7
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
- `apps/gateway/src/federation/server/verbs/search.controller.ts`
|
||||
- `apps/gateway/src/federation/server/audit.service.ts` (async write, no blocking)
|
||||
- `apps/gateway/src/federation/server/rate-limit.guard.ts`
|
||||
- Tests in harness
|
||||
|
||||
**Acceptance tests:**
|
||||
| # | Test | Layer |
|
||||
| - | ------------------------------------------------------------------------------------------------- | ----------- |
|
||||
| 1 | `search` returns ranked hits only from allowed resources | E2E |
|
||||
| 2 | `search` excluding `credentials` does not return a match even when keyword matches a credential name | E2E |
|
||||
| 3 | Every successful request appears in `federation_audit_log` within 1s | Integration |
|
||||
| 4 | Denied request (403) is also audited with `outcome='denied'` | Integration |
|
||||
| 5 | Audit row stores query hash but NOT query body | Unit |
|
||||
| 6 | 61st request in 60s window returns 429 with `Retry-After` | E2E |
|
||||
| 7 | Per-grant override (e.g., 600/min) takes effect without restart | Integration |
|
||||
| 8 | Audit writes are async: request latency unchanged when audit write slow (simulated) | Integration |
|
||||
|
||||
**Estimated budget:** ~20K tokens
|
||||
**Risk notes:** Ensure audit writes can't block or error-out the request path; use a bounded queue and drop-with-counter pattern rather than in-line writes.
|
||||
|
||||
---
|
||||
|
||||
## M5 — Cache + Offline Degradation + Observability
|
||||
|
||||
**Goal:** Sessions feel fast and stay useful when the peer is slow or down.
|
||||
|
||||
**Scope:**
|
||||
|
||||
- In-memory response cache keyed by `(grant_id, verb, resource, query_hash)`, TTL 30s default
|
||||
- Cache NOT used for `search`; only `list` and `get`
|
||||
- Cache flushed on cert rotation and grant revocation
|
||||
- Circuit breaker per peer: after N failures, fast-fail for cooldown window
|
||||
- `_source` tagging extended with `_cached: true` when served from cache
|
||||
- Agent-visible "federation offline for `<peer>`" signal emitted once per session per peer
|
||||
- OTEL spans: `federation.request` with attrs `grant_id`, `peer`, `verb`, `resource`, `outcome`, `latency_ms`, `cached`
|
||||
- W3C `traceparent` propagated across the mTLS boundary (both directions)
|
||||
- `mosaic federation status` CLI subcommand
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
- `apps/gateway/src/federation/client/response-cache.service.ts`
|
||||
- `apps/gateway/src/federation/client/circuit-breaker.service.ts`
|
||||
- `apps/gateway/src/federation/observability/` (span helpers)
|
||||
- `packages/mosaic/src/commands/federation/status.ts`
|
||||
|
||||
**Acceptance tests:**
|
||||
| # | Test | Layer |
|
||||
| - | --------------------------------------------------------------------------------------------- | ----- |
|
||||
| 1 | Two identical `list` calls within 30s: second served from cache, flagged `_cached` | Integration |
|
||||
| 2 | `search` is never cached: two identical searches both hit the peer | Integration |
|
||||
| 3 | After grant revocation, peer's cache is flushed immediately | Integration |
|
||||
| 4 | After N consecutive failures, circuit opens; subsequent requests fail-fast without network call | E2E |
|
||||
| 5 | Circuit closes after cooldown and next success | E2E |
|
||||
| 6 | With peer offline, session completes using local data, one "federation offline" signal surfaced | E2E |
|
||||
| 7 | OTEL traces show spans on both gateways correlated by `traceparent` | E2E |
|
||||
| 8 | `mosaic federation status` prints peer state, cert expiry, last success/failure, circuit state | Unit |
|
||||
|
||||
**Estimated budget:** ~20K tokens
|
||||
**Risk notes:** Caching correctness under revocation must be provable — write tests that intentionally race revocation against cached hits.
|
||||
|
||||
---
|
||||
|
||||
## M6 — Revocation, Auto-Renewal, CRL
|
||||
|
||||
**Goal:** Grant lifecycle works end-to-end: admin revoke, revoke-on-delete, automatic cert renewal, CRL distribution.
|
||||
|
||||
**Scope:**
|
||||
|
||||
- `mosaic federation grant revoke <id>` → status `revoked`, CRL updated, audit entry
|
||||
- DB hook: deleting a user cascades `revoke-on-delete` on all grants where that user is subject
|
||||
- Step-CA CRL endpoint exposed; serving gateway enforces CRL check on every handshake (cached CRL, refresh interval 60s)
|
||||
- Client-side cert renewal job: at T-7 days, submit renewal CSR; rotate cert atomically; flush cache
|
||||
- On renewal failure, peer marked `degraded` and admin-visible alert emitted
|
||||
- Server A detects revocation on next request (TLS handshake fails with specific error) → peer marked `revoked`, user notified
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
- `apps/gateway/src/federation/server/crl.service.ts` + endpoint
|
||||
- `apps/gateway/src/federation/server/revocation.service.ts`
|
||||
- DB cascade trigger or ORM hook for user deletion → grant revocation
|
||||
- `apps/gateway/src/federation/client/renewal.job.ts` (scheduled)
|
||||
- `packages/mosaic/src/commands/federation/grant.ts` gains `revoke` subcommand
|
||||
|
||||
**Acceptance tests:**
|
||||
| # | Test | Layer |
|
||||
| - | ----------------------------------------------------------------------------------------- | ----- |
|
||||
| 1 | Admin `grant revoke` → A's next request fails with TLS-level error | E2E |
|
||||
| 2 | Deleting subject user on B auto-revokes all grants where that user was the subject | Integration |
|
||||
| 3 | CRL endpoint serves correct list; revoked cert present | Integration |
|
||||
| 4 | Server rejects cert listed in CRL even if cert itself is still time-valid | E2E |
|
||||
| 5 | Cert at T-7 days triggers renewal job; new cert issued and installed without dropped requests | E2E |
|
||||
| 6 | Renewal failure marks peer `degraded` and surfaces alert | Integration |
|
||||
| 7 | A marks peer `revoked` after a revocation-caused handshake failure (not on transient network errors) | E2E |
|
||||
|
||||
**Estimated budget:** ~20K tokens
|
||||
**Risk notes:** The atomic cert swap during renewal is the sharpest edge here — any in-flight request mid-swap must either complete on old or retry on new, never fail mid-call.
|
||||
|
||||
---
|
||||
|
||||
## M7 — Multi-User RBAC Hardening + Team-Scoped Grants + Acceptance Suite
|
||||
|
||||
**Goal:** The full multi-tenant scenario from §4 user stories works end-to-end, with no cross-user leakage under any circumstance.
|
||||
|
||||
**Scope:**
|
||||
|
||||
- Three-user scenario on Server B (E1, E2, E3) each with their own Server A
|
||||
- Team-scoped grants exercised: each employee's team-data visible on their own A, but E1's personal data never visible on E2's A
|
||||
- User-facing UI surfaces on both gateways for: peer list, grant list, audit log viewer, scope editor
|
||||
- Negative-path test matrix (every denial path from PRD §8)
|
||||
- All PRD §15 acceptance criteria mapped to automated tests in the harness
|
||||
- Security review: cert-spoofing, scope-bypass, audit-bypass paths explicitly tested
|
||||
- Cold-storage rollover for audit log >90 days
|
||||
- Docs: operator runbook, onboarding guide, troubleshooting guide
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
- Full federation acceptance suite in `tools/federation-harness/acceptance/`
|
||||
- `apps/web` surfaces for peer/grant/audit management
|
||||
- `docs/federation/RUNBOOK.md`, `docs/federation/ONBOARDING.md`, `docs/federation/TROUBLESHOOTING.md`
|
||||
- Audit cold-tier job (daily cron, moves rows >90d to separate table or object storage)
|
||||
|
||||
**Acceptance tests:**
|
||||
Every PRD §15 criterion must be automated and green. Additionally:
|
||||
|
||||
| # | Test | Layer |
|
||||
| --- | ----------------------------------------------------------------------------------------------------- | ---------------- |
|
||||
| 1 | 3-employee scenario: each A sees only its user's data from B | E2E |
|
||||
| 2 | Grant with team scope returns team data; same grant denied access to another employee's personal data | E2E |
|
||||
| 3 | Concurrent sessions from E1's and E2's Server A to B interleave without any leakage | E2E |
|
||||
| 4 | Audit log across 3-user test shows per-grant trails with no mis-attributed rows | E2E |
|
||||
| 5 | Scope editor UI round-trip: edit → save → next request uses new scope | E2E |
|
||||
| 6 | Attempt to use a revoked grant's cert against a different grant's endpoint: rejected | E2E |
|
||||
| 7 | 90-day-old audit rows moved to cold tier; queryable via explicit historical query | Integration |
|
||||
| 8 | Runbook steps validated: an operator following the runbook can onboard, rotate, and revoke | Manual checklist |
|
||||
|
||||
**Estimated budget:** ~25K tokens
|
||||
**Risk notes:** This is the security-critical milestone. Budget review time here is non-negotiable — plan for two independent code reviews (internal + security-focused) before merge.
|
||||
|
||||
---
|
||||
|
||||
## Total Budget & Timeline Sketch
|
||||
|
||||
| Milestone | Tokens (est.) | Can parallelize? |
|
||||
| --------- | ------------- | ---------------------- |
|
||||
| M1 | 20K | No (foundation) |
|
||||
| M2 | 30K | No (needs M1) |
|
||||
| M3 | 40K | No (needs M2) |
|
||||
| M4 | 20K | No (needs M3) |
|
||||
| M5 | 20K | Yes (with M6 after M4) |
|
||||
| M6 | 20K | Yes (with M5 after M3) |
|
||||
| M7 | 25K | No (needs all) |
|
||||
| **Total** | **~175K** | |
|
||||
|
||||
Parallelization of M5 and M6 after M4 saves one milestone's worth of serial time.
|
||||
|
||||
---
|
||||
|
||||
## Exit Criteria (federation feature complete)
|
||||
|
||||
All of the following must be green on `main`:
|
||||
|
||||
- Every PRD §15 acceptance criterion automated and passing
|
||||
- Every milestone's acceptance table green
|
||||
- Security review sign-off on M7
|
||||
- Runbook walk-through completed by operator (not author)
|
||||
- `mosaic doctor` recognizes federated tier and reports peer health accurately
|
||||
- Two-gateway production deployment (woltje.com ↔ uscllc.com) operational for ≥7 days without incident
|
||||
|
||||
---
|
||||
|
||||
## Next Step After This Doc Is Approved
|
||||
|
||||
1. File tracking issues on `git.mosaicstack.dev/mosaicstack/stack` — one per milestone, labeled `epic:federation`
|
||||
2. Populate `docs/TASKS.md` with M1's task breakdown (per-task agent assignment, budget, dependencies)
|
||||
3. Begin M1 implementation
|
||||
@@ -0,0 +1,108 @@
|
||||
# Mission Manifest — Federation v1
|
||||
|
||||
> Persistent document tracking full mission scope, status, and session history.
|
||||
> Updated by the orchestrator at each phase transition and milestone completion.
|
||||
|
||||
## Mission
|
||||
|
||||
**ID:** federation-v1-20260419
|
||||
**Statement:** Jarvis operates across 3–4 workstations in two physical locations (home, USC). The user currently reaches back to a single jarvis-brain checkout from every session; a prior OpenBrain attempt caused cache, latency, and opacity pain. This mission builds asymmetric federation between Mosaic Stack gateways so that a session on a user's home gateway can query their work gateway in real time without data ever persisting across the boundary, with full multi-tenant isolation and standard-PKI (X.509 / Step-CA) trust management.
|
||||
**Phase:** M3 active — mTLS handshake + list/get/capabilities verbs + scope enforcement
|
||||
**Current Milestone:** FED-M3
|
||||
**Progress:** 2 / 7 milestones
|
||||
**Status:** active
|
||||
**Last Updated:** 2026-04-21 (M2 closed via PR #503, tag `fed-v0.2.0-m2`, issue #461 closed; M3 decomposed into 14 tasks)
|
||||
**Parent Mission:** None — new mission
|
||||
|
||||
## Test Infrastructure
|
||||
|
||||
| Host | Role | Image | Tier |
|
||||
| ----------------------- | ----------------------------------- | ------------------------------------- | --------- |
|
||||
| `mos-test-1.woltje.com` | Federation Server A (querying side) | `gateway:fed-v0.1.0-m1` (M1 baseline) | federated |
|
||||
| `mos-test-2.woltje.com` | Federation Server B (serving side) | `gateway:fed-v0.1.0-m1` (M1 baseline) | federated |
|
||||
|
||||
These are TEST hosts for federation E2E (M3+). Distinct from PRD AC-12 production targets (`woltje.com` ↔ `uscllc.com`). Deployment workstream tracked in `docs/federation/TASKS.md` under FED-M2-DEPLOY-\*.
|
||||
|
||||
## Context
|
||||
|
||||
Federation is the solution to what originally drove OpenBrain. The prior attempt coupled every agent session to a remote service, introduced cache/latency/opacity pain, and created a hard dependency that punished offline use. This redesign:
|
||||
|
||||
1. Makes federation **gateway-to-gateway**, not agent-to-service
|
||||
2. Keeps each user's home instance as source of truth for their data
|
||||
3. Exposes scoped, read-only data on demand without persisting across the boundary
|
||||
4. Uses X.509 mTLS via Step-CA so rotation/revocation/CRL/OCSP are standard
|
||||
5. Supports multi-tenant serving sides (employees on uscllc.com each federating back to their own home gateway) with no cross-user leakage
|
||||
6. Requires federation-tier instances on both sides (PG + pgvector + Valkey) — local/standalone tiers cannot federate
|
||||
7. Works over public HTTPS (no VPN required); Tailscale is an optional overlay
|
||||
|
||||
Key design references:
|
||||
|
||||
- `docs/federation/PRD.md` — 16-section product requirements
|
||||
- `docs/federation/MILESTONES.md` — 7-milestone decomposition with per-milestone acceptance tests
|
||||
- `docs/federation/TASKS.md` — per-task breakdown (M1 populated; M2-M7 deferred to mission planning)
|
||||
- `docs/research/mempalace-evaluation/` (in jarvis-brain) — why we didn't adopt MemPalace
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] AC-1: Two Mosaic Stack gateways on different hosts can establish a federation grant via CLI-driven onboarding
|
||||
- [ ] AC-2: Server A can query Server B for `tasks`, `notes`, `memory` respecting scope filters
|
||||
- [ ] AC-3: User on B with no grant cannot be queried by A, even if A has a valid grant for another user (cross-user isolation)
|
||||
- [ ] AC-4: Revoking a grant on B causes A's next request to fail with a clear error within one request cycle
|
||||
- [ ] AC-5: Cert rotation happens automatically at T-7 days; in-progress session survives rotation without user action
|
||||
- [ ] AC-6: Rate-limit enforcement returns 429 with `Retry-After`; client backs off
|
||||
- [ ] AC-7: With B unreachable, a session on A completes using local data and surfaces "federation offline for `<peer>`" once per session
|
||||
- [ ] AC-8: Every federated request appears in B's `federation_audit_log` within 1 second
|
||||
- [ ] AC-9: Scope excluding `credentials` means credentials are never returned — even via `search` with matching keywords
|
||||
- [ ] AC-10: `mosaic federation status` shows cert expiry, grant status, last success/failure per peer
|
||||
- [ ] AC-11: Full 3-employee multi-tenant scenario passes with no cross-user leakage
|
||||
- [ ] AC-12: Two-gateway production deployment (woltje.com ↔ uscllc.com) operational ≥7 days without incident
|
||||
- [ ] AC-13: All 7 milestones ship as merged PRs with green CI and closed issues
|
||||
|
||||
## Milestones
|
||||
|
||||
| # | ID | Name | Status | Branch | Issue | Started | Completed |
|
||||
| --- | ------ | --------------------------------------------- | ----------- | ------------------ | ----- | ---------- | ---------- |
|
||||
| 1 | FED-M1 | Federated tier infrastructure | done | (12 PRs #470-#481) | #460 | 2026-04-19 | 2026-04-19 |
|
||||
| 2 | FED-M2 | Step-CA + grant schema + admin CLI | done | (PRs #483-#503) | #461 | 2026-04-21 | 2026-04-21 |
|
||||
| 3 | FED-M3 | mTLS handshake + list/get + scope enforcement | in-progress | (decomposition) | #462 | 2026-04-21 | — |
|
||||
| 4 | FED-M4 | search verb + audit log + rate limit | not-started | — | #463 | — | — |
|
||||
| 5 | FED-M5 | Cache + offline degradation + OTEL | not-started | — | #464 | — | — |
|
||||
| 6 | FED-M6 | Revocation + auto-renewal + CRL | not-started | — | #465 | — | — |
|
||||
| 7 | FED-M7 | Multi-user RBAC hardening + acceptance suite | not-started | — | #466 | — | — |
|
||||
|
||||
## Budget
|
||||
|
||||
| Milestone | Est. tokens | Parallelizable? |
|
||||
| --------- | ----------- | ---------------------- |
|
||||
| FED-M1 | 20K | No (foundation) |
|
||||
| FED-M2 | 30K | No (needs M1) |
|
||||
| FED-M3 | 40K | No (needs M2) |
|
||||
| FED-M4 | 20K | No (needs M3) |
|
||||
| FED-M5 | 20K | Yes (with M6 after M4) |
|
||||
| FED-M6 | 20K | Yes (with M5 after M3) |
|
||||
| FED-M7 | 25K | No (needs all) |
|
||||
| **Total** | **~175K** | |
|
||||
|
||||
## Session History
|
||||
|
||||
| Session | Date | Runtime | Outcome |
|
||||
| ------- | ----------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| S1 | 2026-04-19 | claude | PRD authored, MILESTONES decomposed, 7 issues filed |
|
||||
| S2-S4 | 2026-04-19 | claude | FED-M1 complete: 12 tasks (PRs #470-#481) merged; tag `fed-v0.1.0-m1` |
|
||||
| S5-S22 | 2026-04-19 → 2026-04-21 | claude | FED-M2 complete: 13 tasks (PRs #483-#503) merged; tag `fed-v0.2.0-m2`; issue #461 closed. Step-CA + grant schema + admin CLI shipped. |
|
||||
| S23 | 2026-04-21 | claude | M3 decomposed into 14 tasks in `docs/federation/TASKS.md`. Manifest M3 row → in-progress. Next: kickoff M3-01. |
|
||||
|
||||
## Next Step
|
||||
|
||||
FED-M3 active. Decomposition landed in `docs/federation/TASKS.md` (M3-01..M3-14, ~100K estimate). Tracking issue #462.
|
||||
|
||||
Execution plan (parallel where possible):
|
||||
|
||||
- **Foundation**: M3-01 (DTOs in `packages/types/src/federation/`) starts immediately — sonnet subagent on `feat/federation-m3-types`. Blocks all server + client work.
|
||||
- **Server stream** (after M3-01): M3-03 (AuthGuard) + M3-04 (ScopeService) in series, then M3-05 / M3-06 / M3-07 (verbs) in parallel.
|
||||
- **Client stream** (after M3-01, parallel with server): M3-08 (FederationClient) → M3-09 (QuerySourceService).
|
||||
- **Harness** (parallel with everything): M3-02 (`tools/federation-harness/`) — needed for M3-11.
|
||||
- **Test gates**: M3-10 (Integration) → M3-11 (E2E with harness) → M3-12 (Independent security review, two rounds budgeted).
|
||||
- **Close**: M3-13 (Docs) → M3-14 (release tag `fed-v0.3.0-m3`, close #462).
|
||||
|
||||
**Test-bed fallback:** `mos-test-1/-2` deploy is still blocked on `FED-M2-DEPLOY-IMG-FIX`. The harness in M3-02 ships a local two-gateway docker-compose so M3-11 is not blocked. Production-host validation is M7's responsibility (PRD AC-12).
|
||||
@@ -0,0 +1,330 @@
|
||||
# Mosaic Stack — Federation PRD
|
||||
|
||||
**Status:** Draft v1 (locked for implementation)
|
||||
**Owner:** Jason
|
||||
**Date:** 2026-04-19
|
||||
**Scope:** Enables cross-instance data federation between Mosaic Stack gateways with asymmetric trust, multi-tenant scoping, and no cross-boundary data persistence.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem Statement
|
||||
|
||||
Jarvis operates across 3–4 workstations in two physical locations (home, USC). The user currently reaches back to a single jarvis-brain checkout from every session, and has tried OpenBrain to solve cross-session state — with poor results (cache invalidation, latency, opacity, hard dependency on a remote service).
|
||||
|
||||
The goal is a federation model where each user's **home instance** remains the source of truth for their personal data, and **work/shared instances** expose scoped data to that user's home instance on demand — without persisting anything across the boundary.
|
||||
|
||||
## 2. Goals
|
||||
|
||||
1. A user logged into their **home gateway** (Server A) can query their **work gateway** (Server B) in real time during a session.
|
||||
2. Data returned from Server B is used in-session only; never written to Server A storage.
|
||||
3. Server B has multiple users, each with their own Server A. No user's data leaks to another user.
|
||||
4. Federation works over public HTTPS (no VPN required). Tailscale is a supported optional overlay.
|
||||
5. Sync latency target: seconds, or at the next data need of the agent.
|
||||
6. Graceful degradation: if the remote instance is unreachable, the local session continues with local data and a clear "federation offline" signal.
|
||||
7. Teams exist on both sides. A federation grant can share **team-owned** data without exposing other team members' personal data.
|
||||
8. Auth and revocation use standard PKI (X.509) so that certificate tooling (Step-CA, rotation, OCSP, CRL) is available out of the box.
|
||||
|
||||
## 3. Non-Goals (v1)
|
||||
|
||||
- Mesh federation (N-to-N). v1 is strictly A↔B pairs.
|
||||
- Cross-instance writes. All federation is **read-only** on the remote side.
|
||||
- Shared agent sessions across instances. Sessions live on one instance; federation is data-plane only.
|
||||
- Cross-instance SSO. Each instance owns its own BetterAuth identity store; federation is service-to-service, not user-to-user.
|
||||
- Realtime push from B→A. v1 is pull-only (A pulls from B during a session).
|
||||
- Global search index. Federation is query-by-query, not index replication.
|
||||
|
||||
## 4. User Stories
|
||||
|
||||
- **US-1 (Solo user at home):** As the sole user on Server A, I want my agent session on workstation-1 to see the same data it saw on workstation-2, without running OpenBrain.
|
||||
- **US-2 (Cross-location):** As a user with a home server and a work server, I want a session on my home laptop to transparently pull my USC-owned tasks/notes when I ask for them.
|
||||
- **US-3 (Work admin):** As the admin of mosaic.uscllc.com, I want to grant each employee's home gateway scoped read access to only their own data plus explicitly-shared team data.
|
||||
- **US-4 (Privacy boundary):** As employee A on mosaic.uscllc.com, my data must never appear in a session on employee B's home gateway — even if both are federated with uscllc.com.
|
||||
- **US-5 (Revocation):** As a work admin, when I delete an employee, their home gateway loses access within one request cycle.
|
||||
- **US-6 (Offline):** As a user in a hotel with flaky wifi, my local session keeps working; federation calls fail fast and are reported as "offline," not hung.
|
||||
|
||||
## 5. Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐ mTLS / X.509 ┌─────────────────────────────────────┐
|
||||
│ Server A — mosaic.woltje.com │ ───────────────────────► │ Server B — mosaic.uscllc.com │
|
||||
│ (home, master for Jason) │ ◄── JSON over HTTPS │ (work, multi-tenant) │
|
||||
│ │ │ │
|
||||
│ ┌──────────────┐ ┌──────────────┐ │ │ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ Gateway │ │ Postgres │ │ │ │ Gateway │ │ Postgres │ │
|
||||
│ │ (NestJS) │──│ (local SSOT)│ │ │ │ (NestJS) │──│ (tenant SSOT)│ │
|
||||
│ └──────┬───────┘ └──────────────┘ │ │ └──────┬───────┘ └──────────────┘ │
|
||||
│ │ │ │ │ │
|
||||
│ │ FederationClient │ │ │ FederationServer │
|
||||
│ │ (outbound, scoped query) │ │ │ (inbound, RBAC-gated) │
|
||||
│ └───────────────────────────┼──────────────────────────┼────────┘ │
|
||||
│ │ │ │
|
||||
│ Step-CA (issues A's client cert) │ │ Step-CA (issues B's server cert, │
|
||||
│ │ │ trusts A's CA root on grant)│
|
||||
└─────────────────────────────────────┘ └──────────────────────────────────────┘
|
||||
```
|
||||
|
||||
- Federation is a **transport-layer** concern between two gateways, implemented as a new internal module on each gateway.
|
||||
- Both sides run the same code. Direction (client vs. server role) is per-request.
|
||||
- Nothing in the agent runtime changes — agents query the gateway; the gateway decides local vs. remote.
|
||||
|
||||
## 6. Transport & Authentication
|
||||
|
||||
**Transport:** HTTPS with mutual TLS (mTLS).
|
||||
|
||||
**Identity:** X.509 client certificates issued by Step-CA. Each federation grant materializes as a client cert on the requesting side and a trust-anchor entry (CA root or explicit cert) on the serving side.
|
||||
|
||||
**Why mTLS over HMAC bearer tokens:**
|
||||
|
||||
- Standard rotation/revocation semantics (renew, CRL, OCSP).
|
||||
- The cert subject carries identity claims (user, grant_id) that don't need a separate DB lookup to verify authenticity.
|
||||
- Client certs never transit request bodies, so they can't be logged by accident.
|
||||
- Transport is pinned at the TLS layer, not re-validated per-handler.
|
||||
|
||||
**Cert contents (SAN + subject):**
|
||||
|
||||
- `CN=grant-<uuid>`
|
||||
- `O=<requesting-server-hostname>` (e.g., `mosaic.woltje.com`)
|
||||
- Custom OIDs embedded in SAN otherName:
|
||||
- `mosaic.federation.grantId` (UUID)
|
||||
- `mosaic.federation.subjectUserId` (user on the **serving** side that this grant acts-as)
|
||||
- Default lifetime: **30 days**, with auto-renewal at T-7 days if the grant is still active.
|
||||
|
||||
**Step-CA topology (v1):** Each server runs its own Step-CA instance. During onboarding, the serving side imports the requesting side's CA root. A central/shared Step-CA is out of scope for v1.
|
||||
|
||||
**Handshake:**
|
||||
|
||||
1. Client (A) opens HTTPS to B with its grant cert.
|
||||
2. B validates cert chain against trusted CA roots for that grant.
|
||||
3. B extracts `grantId` and `subjectUserId` from the cert.
|
||||
4. B loads the grant record, checks it is `active`, not revoked, and not expired.
|
||||
5. B enforces scope and rate-limit for this grant.
|
||||
6. Request proceeds; response returned.
|
||||
|
||||
## 7. Data Model
|
||||
|
||||
All tables live on **each instance's own Postgres**. Federation grants are bilateral — each side has a record of the grant.
|
||||
|
||||
### 7.1 `federation_grants` (on serving side, Server B)
|
||||
|
||||
| Field | Type | Notes |
|
||||
| --------------------------- | ----------- | ------------------------------------------------- |
|
||||
| `id` | uuid PK | |
|
||||
| `subject_user_id` | uuid FK | Which local user this grant acts-as |
|
||||
| `requesting_server` | text | Hostname of requesting gateway (e.g., woltje.com) |
|
||||
| `requesting_ca_fingerprint` | text | SHA-256 of trusted CA root |
|
||||
| `active_cert_fingerprint` | text | SHA-256 of currently valid client cert |
|
||||
| `scope` | jsonb | See §8 |
|
||||
| `rate_limit_rpm` | int | Default 60 |
|
||||
| `status` | enum | `pending`, `active`, `suspended`, `revoked` |
|
||||
| `created_at` | timestamptz | |
|
||||
| `activated_at` | timestamptz | |
|
||||
| `revoked_at` | timestamptz | |
|
||||
| `last_used_at` | timestamptz | |
|
||||
| `notes` | text | Admin-visible description |
|
||||
|
||||
### 7.2 `federation_peers` (on requesting side, Server A)
|
||||
|
||||
| Field | Type | Notes |
|
||||
| --------------------- | ----------- | ------------------------------------------------ |
|
||||
| `id` | uuid PK | |
|
||||
| `peer_hostname` | text | e.g., `mosaic.uscllc.com` |
|
||||
| `peer_ca_fingerprint` | text | SHA-256 of peer's CA root |
|
||||
| `grant_id` | uuid | The grant ID assigned by the peer |
|
||||
| `local_user_id` | uuid FK | Who on Server A this federation belongs to |
|
||||
| `client_cert_pem` | text (enc) | Current client cert (PEM); rotated automatically |
|
||||
| `client_key_pem` | text (enc) | Private key (encrypted at rest) |
|
||||
| `cert_expires_at` | timestamptz | |
|
||||
| `status` | enum | `pending`, `active`, `degraded`, `revoked` |
|
||||
| `last_success_at` | timestamptz | |
|
||||
| `last_failure_at` | timestamptz | |
|
||||
| `notes` | text | |
|
||||
|
||||
### 7.3 `federation_audit_log` (on serving side, Server B)
|
||||
|
||||
| Field | Type | Notes |
|
||||
| ------------- | ----------- | ------------------------------------------------ |
|
||||
| `id` | uuid PK | |
|
||||
| `grant_id` | uuid FK | |
|
||||
| `occurred_at` | timestamptz | indexed |
|
||||
| `verb` | text | `query`, `handshake`, `rejected`, `rate_limited` |
|
||||
| `resource` | text | e.g., `tasks`, `notes`, `credentials` |
|
||||
| `query_hash` | text | SHA-256 of normalized query (no payload stored) |
|
||||
| `outcome` | text | `ok`, `denied`, `error` |
|
||||
| `bytes_out` | int | |
|
||||
| `latency_ms` | int | |
|
||||
|
||||
**Audit policy:** Every federation request is logged on the serving side. Read-only requests only — no body capture. Retention: 90 days hot, then roll to cold storage.
|
||||
|
||||
## 8. RBAC & Scope
|
||||
|
||||
Every federation grant has a scope object that answers three questions for every inbound request:
|
||||
|
||||
1. **Who is acting?** — `subject_user_id` from the cert.
|
||||
2. **What resources?** — an allowlist of resource types (`tasks`, `notes`, `credentials`, `memory`, `teams/:id/tasks`, …).
|
||||
3. **Filter expression** — predicates applied on top of the subject's normal RBAC (see below).
|
||||
|
||||
### 8.1 Scope schema
|
||||
|
||||
```json
|
||||
{
|
||||
"resources": ["tasks", "notes", "memory"],
|
||||
"filters": {
|
||||
"tasks": { "include_teams": ["team_uuid_1", "team_uuid_2"], "include_personal": true },
|
||||
"notes": { "include_personal": true, "include_teams": [] },
|
||||
"memory": { "include_personal": true }
|
||||
},
|
||||
"excluded_resources": ["credentials", "api_keys"],
|
||||
"max_rows_per_query": 500
|
||||
}
|
||||
```
|
||||
|
||||
### 8.2 Access rule (enforced on serving side)
|
||||
|
||||
For every inbound federated query on resource R:
|
||||
|
||||
1. Resolve effective identity → `subject_user_id`.
|
||||
2. Check R is in `scope.resources` and NOT in `scope.excluded_resources`. Otherwise 403.
|
||||
3. Evaluate the user's **normal RBAC** (what would they see if they logged into Server B directly)?
|
||||
4. Intersect with the scope filter (e.g., only team X, only personal).
|
||||
5. Apply `max_rows_per_query`.
|
||||
6. Return; log to audit.
|
||||
|
||||
### 8.3 Team boundary guarantees
|
||||
|
||||
- Scope filters are additive, never subtractive of the native RBAC. A grant cannot grant access the user would not have had themselves.
|
||||
- `include_teams` means "only these teams," not "these teams in addition to all teams."
|
||||
- `include_personal: false` hides the user's personal data entirely from federation, even if they own it — useful for work-only accounts.
|
||||
|
||||
### 8.4 No cross-user leakage
|
||||
|
||||
When Server B has multiple users (employees) all federating back to their own Server A:
|
||||
|
||||
- Each employee has their own grant with their own `subject_user_id`.
|
||||
- The cert is bound to a specific grant; there is no mechanism by which one grant's cert can be used to impersonate another.
|
||||
- Audit log is per-grant.
|
||||
|
||||
## 9. Query Model
|
||||
|
||||
Federation exposes a **narrow read API**, not arbitrary SQL.
|
||||
|
||||
### 9.1 Supported verbs (v1)
|
||||
|
||||
| Verb | Purpose | Returns |
|
||||
| -------------- | ------------------------------------------ | ------------------------------- |
|
||||
| `list` | Paginated list of a resource type | Array of resources |
|
||||
| `get` | Fetch a single resource by id | One resource or 404 |
|
||||
| `search` | Keyword search within allowed resources | Ranked list of hits |
|
||||
| `capabilities` | What this grant is allowed to do right now | Scope object + rate-limit state |
|
||||
|
||||
### 9.2 Not in v1
|
||||
|
||||
- Write verbs.
|
||||
- Aggregations / analytics.
|
||||
- Streaming / subscriptions (future: see §13).
|
||||
|
||||
### 9.3 Agent-facing integration
|
||||
|
||||
Agents never call federation directly. Instead:
|
||||
|
||||
- The gateway query layer accepts `source: "local" | "federated:<peer_hostname>" | "all"`.
|
||||
- `"all"` fans out in parallel, merges results, tags each with `_source`.
|
||||
- Federation results are in-memory only; the gateway does not persist them.
|
||||
|
||||
## 10. Caching
|
||||
|
||||
- **In-memory response cache** with short TTL (default 30s) for `list` and `get`. `search` is not cached.
|
||||
- Cache is keyed by `(grant_id, verb, resource, query_hash)`.
|
||||
- Cache is flushed on cert rotation and on grant revocation.
|
||||
- No disk cache. No cross-session cache.
|
||||
|
||||
## 11. Bootstrap & Onboarding
|
||||
|
||||
### 11.1 Instance capability tiers
|
||||
|
||||
| Tier | Storage | Queue | Memory | Can federate? |
|
||||
| ------------ | -------- | ------- | -------- | --------------------- |
|
||||
| `local` | PGlite | in-proc | keyword | No |
|
||||
| `standalone` | Postgres | Valkey | keyword | No (can be client) |
|
||||
| `federated` | Postgres | Valkey | pgvector | Yes (server + client) |
|
||||
|
||||
Federation requires `federated` tier on **both** sides.
|
||||
|
||||
### 11.2 Onboarding flow (admin-driven)
|
||||
|
||||
1. Admin on Server B runs `mosaic federation grant create --user <user-id> --peer <peer-hostname> --scope-file scope.json`.
|
||||
2. Server B generates a `grant_id`, prints a one-time enrollment URL containing the grant ID + B's CA root fingerprint.
|
||||
3. Admin on Server A (or the user themselves, if allowed) runs `mosaic federation peer add <enrollment-url>`.
|
||||
4. Server A's Step-CA generates a CSR for the new grant. A submits the CSR to B over a short-lived enrollment endpoint (single-use token in the enrollment URL).
|
||||
5. B's Step-CA signs the cert (with grant ID embedded in SAN OIDs), returns it.
|
||||
6. A stores the signed cert + private key (encrypted) in `federation_peers`.
|
||||
7. Grant status flips from `pending` to `active` on both sides.
|
||||
8. Cert auto-renews at T-7 days using the standard Step-CA renewal flow as long as the grant remains active.
|
||||
|
||||
### 11.3 Revocation
|
||||
|
||||
- **Admin-initiated:** `mosaic federation grant revoke <grant-id>` on B flips status to `revoked`, adds the cert to B's CRL, and writes an audit entry.
|
||||
- **Revoke-on-delete:** Deleting a user on B automatically revokes all grants where that user is the subject.
|
||||
- Server A learns of revocation on the next request (TLS handshake fails) and flips the peer to `revoked`.
|
||||
|
||||
### 11.4 Rate limit
|
||||
|
||||
Default `60 req/min` per grant. Configurable per grant. Enforced at the serving side. A rate-limited request returns `429` with `Retry-After`.
|
||||
|
||||
## 12. Operational Concerns
|
||||
|
||||
- **Observability:** Each federation request emits an OTEL span with `grant_id`, `peer`, `verb`, `resource`, `outcome`, `latency_ms`. Traces correlate across both servers via W3C traceparent.
|
||||
- **Health check:** `mosaic federation status` on each side shows active grants, last-success times, cert expirations, and any CRL mismatches.
|
||||
- **Backpressure:** If the serving side is overloaded, it returns `503` with a structured body; the client marks the peer `degraded` and falls back to local-only until the next successful handshake.
|
||||
- **Secrets:** `client_key_pem` in `federation_peers` is encrypted with the gateway's key (sealed with the instance's master key — same mechanism as `provider_credentials`).
|
||||
- **Credentials never cross:** The `credentials` resource type is in the default excluded list. It must be explicitly added to scope (admin action, logged) and even then is per-grant and per-user.
|
||||
|
||||
## 13. Future (post-v1)
|
||||
|
||||
- B→A push (e.g., "notify A when a task assigned to subject changes") via Socket.IO over mTLS.
|
||||
- Mesh (N-to-N) federation.
|
||||
- Write verbs with conflict resolution.
|
||||
- Shared Step-CA (a "root of roots") so that onboarding doesn't require exchanging CA roots.
|
||||
- Federated memory search over vector indexes with homomorphic filtering.
|
||||
|
||||
## 14. Locked Decisions (was "Open Questions")
|
||||
|
||||
| # | Question | Decision |
|
||||
| --- | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 1 | What happens to a grant when its subject user is deleted? | **Revoke-on-delete.** All grants where the user is subject are auto-revoked and CRL'd. |
|
||||
| 2 | Do we audit read-only requests? | **Yes.** All federated reads are audited on the serving side. Bodies are not captured; query hash + metadata only. |
|
||||
| 3 | Default rate limit? | **60 requests per minute per grant,** override-able per grant. |
|
||||
| 4 | How do we verify the requesting-server's identity beyond the grant token? | **X.509 client cert tied to the user,** issued by Step-CA (per-server) or locally generated. Cert subject carries `grantId` + `subjectUserId`. |
|
||||
|
||||
### M1 decisions
|
||||
|
||||
- **Postgres deployment:** **Containerized** alongside the gateway in M1 (Docker Compose profile). Moving to a dedicated host is a M5+ operational concern, not a v1 feature.
|
||||
- **Instance signing key:** **Separate** from the Step-CA key. Step-CA signs federation certs; the instance master key seals at-rest secrets (client keys, provider credentials). Different blast-radius, different rotation cadences.
|
||||
|
||||
## 15. Acceptance Criteria
|
||||
|
||||
- [ ] Two Mosaic Stack gateways on different hosts can establish a federation grant via the CLI-driven onboarding flow.
|
||||
- [ ] Server A can query Server B for `tasks`, `notes`, `memory` respecting scope filters.
|
||||
- [ ] A user on B with no grant cannot be queried by A, even if A has a valid grant for another user.
|
||||
- [ ] Revoking a grant on B causes A's next request to fail with a clear error within one request cycle.
|
||||
- [ ] Cert rotation happens automatically at T-7 days; an in-progress session survives rotation without user action.
|
||||
- [ ] Rate-limit enforcement returns 429 with `Retry-After`; client backs off.
|
||||
- [ ] With B unreachable, a session on A completes using local data and surfaces a "federation offline for `<peer>`" signal once.
|
||||
- [ ] Every federated request appears in B's `federation_audit_log` within 1 second.
|
||||
- [ ] A scope excluding `credentials` means credentials are not returnable even via `search` with matching keywords.
|
||||
- [ ] `mosaic federation status` shows cert expiry, grant status, and last success/failure per peer.
|
||||
|
||||
## 16. Implementation Milestones (reference)
|
||||
|
||||
Milestones live in `docs/federation/MILESTONES.md` (to be authored next). High-level:
|
||||
|
||||
- **M1:** Server A runs `federated` tier standalone (Postgres + Valkey + pgvector, containerized). No peer yet.
|
||||
- **M2:** Step-CA embedded; `federation_grants` / `federation_peers` schema + admin CLI.
|
||||
- **M3:** Handshake + `list`/`get` verbs with scope enforcement.
|
||||
- **M4:** `search` verb, audit log, rate limits.
|
||||
- **M5:** Cache layer, offline-degradation UX, observability surfaces.
|
||||
- **M6:** Revocation flows (admin + revoke-on-delete), cert auto-renewal.
|
||||
- **M7:** Multi-user RBAC hardening on B, team-scoped grants end-to-end, acceptance suite green.
|
||||
|
||||
---
|
||||
|
||||
**Next step after PRD sign-off:** author `docs/federation/MILESTONES.md` with per-milestone acceptance tests and estimated token budget, then file tracking issues on `git.mosaicstack.dev/mosaicstack/stack`.
|
||||
@@ -0,0 +1,74 @@
|
||||
# Federated Tier Setup Guide
|
||||
|
||||
> **KBN-101 N-1 hold:** This page is **non-operative** and grants no current command
|
||||
> authority until KBN-101-00, KBN-101-03, and KBN-101-05 land and KBN-101-08 activates a
|
||||
> reviewed release. It does not authorize a deployment operation, initialization artifacts,
|
||||
> implicit extension/schema/migration creation, raw `CREATE`, direct database initialization, or
|
||||
> a Gateway against an unverified database. The prior direct-start wording is retired; its
|
||||
> regression fixture is owned by KBN-101-06.
|
||||
|
||||
## Held future procedure
|
||||
|
||||
This section is non-operative and grants no current command authority until KBN-101-00, KBN-101-03, and KBN-101-05 land.
|
||||
|
||||
The deployment control plane—not an operator shell or deployment lifecycle hook—performs this
|
||||
exact held future sequence after activation authorization: external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness.
|
||||
|
||||
1. External bootstrap provisions the approved database/extension prerequisites.
|
||||
2. TLS/roles are installed through the generation-pinned renderer.
|
||||
3. The dedicated one-shot runner executes `mosaic-db-migrator --run`.
|
||||
4. The same runner executes `mosaic-db-migrator --verify`, including readiness and the
|
||||
importer-target attestation where that route is enabled.
|
||||
5. Only after successful verification may Gateway reach its independent verified-TLS Gateway
|
||||
readiness gate.
|
||||
|
||||
No step may be reordered, skipped, replaced by a raw SQL command, or delegated to an initialization
|
||||
hook.
|
||||
A missing extension, schema, migration, role, secret generation, or readiness proof is a failed
|
||||
control-plane precondition; it is not an instruction to start Compose, retry startup, or create
|
||||
anything directly.
|
||||
|
||||
## N-1 status and required disposition
|
||||
|
||||
The current branch retains historical federation artifacts, but they are not a deployable
|
||||
procedure. `docs/federation/TASKS.md` records their shipped status only. KBN-101-02 retires
|
||||
runtime/init DDL; KBN-101-05 owns the renderer/deployment handoff; KBN-101-06 verifies the
|
||||
finite scanner and command matrix; and KBN-101-07 owns this operator route. A path named in an
|
||||
inventory, a historical-status label, or a normative requirement cannot suppress the semantic
|
||||
checks above.
|
||||
|
||||
Until the activation certificate names an exact release, use no database startup or recovery
|
||||
command from this document. For the produced importer interface, see
|
||||
[the federated tier migration contract](../guides/migrate-tier.md); it is likewise non-operative
|
||||
until activation.
|
||||
|
||||
## Federation and Step-CA reference
|
||||
|
||||
Federation uses PostgreSQL 17 with pgvector, Valkey, and a shared configuration across multiple
|
||||
Gateway instances. Step-CA issues federation peer X.509 certificates whose custom OIDs carry a
|
||||
grant and subject identity. The following facts are reference material only; provisioning and
|
||||
secret delivery remain deployment-control-plane work under the activation sequence.
|
||||
|
||||
| OID | Name | Description |
|
||||
| ------------------- | ------------------------ | --------------------- |
|
||||
| 1.3.6.1.4.1.99999.1 | `mosaic_grant_id` | Federation grant UUID |
|
||||
| 1.3.6.1.4.1.99999.2 | `mosaic_subject_user_id` | Subject user UUID |
|
||||
|
||||
The internal arc `1.3.6.1.4.1.99999` is development-only. Before an externally reachable
|
||||
production deployment, register an IANA Private Enterprise Number and version the assignments.
|
||||
Each value is DER-encoded as an ASN.1 UTF8String containing the UUID.
|
||||
|
||||
The future activated Gateway requires `STEP_CA_URL`, `STEP_CA_PROVISIONER_PASSWORD`,
|
||||
`STEP_CA_PROVISIONER_KEY_JSON`, `STEP_CA_ROOT_CERT_PATH`, and `BETTER_AUTH_SECRET` through the
|
||||
reviewed secret mechanism. These names do not authorize shell exports, copied credential files,
|
||||
or an ad hoc service start.
|
||||
|
||||
## Failure disposition
|
||||
|
||||
- A TLS, CA, SAN, role, runner, or readiness failure is a control-plane incident. Preserve only
|
||||
sanitized evidence and follow the approved rollback/repair record.
|
||||
- A pgvector/extension failure is a failed external-bootstrap or runner precondition. Do not use
|
||||
direct extension SQL, init artifacts, or a startup retry as remediation.
|
||||
- A port, container, or Valkey problem does not permit bypassing the activation sequence.
|
||||
- Federation peer-key rotation remains deferred until its separately approved migration plan;
|
||||
do not rotate `BETTER_AUTH_SECRET` without that plan.
|
||||
@@ -0,0 +1,154 @@
|
||||
# Tasks — Federation v1
|
||||
|
||||
> Single-writer: orchestrator only. Workers read but never modify.
|
||||
>
|
||||
> **Mission:** federation-v1-20260419
|
||||
> **Schema:** `| id | status | description | issue | agent | branch | depends_on | estimate | notes |`
|
||||
> **Status values:** `not-started` | `in-progress` | `done` | `blocked` | `failed` | `needs-qa`
|
||||
> **Agent values:** `codex` | `glm-5.1` | `haiku` | `sonnet` | `opus` | `—` (auto)
|
||||
>
|
||||
> **Scope of this file:** M1 is fully decomposed below. M2–M7 are placeholders pending each milestone's entry into active planning — the orchestrator expands them one milestone at a time to avoid speculative decomposition of work whose shape will depend on what M1 surfaces.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 1 — Federated tier infrastructure (FED-M1)
|
||||
|
||||
Goal: Gateway runs in `federated` tier with containerized PG+pgvector+Valkey. No federation logic yet. Existing standalone behavior does not regress.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ---------------------------------- | ---------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| FED-M1-01 | done | Extend `mosaic.config.json` schema: add `"federated"` to `tier` enum in validator + TS types. Keep `local` and `standalone` working. Update schema docs/README where referenced. | #460 | sonnet | feat/federation-m1-tier-config | — | 4K | Shipped in PR #470. Renamed `team` → `standalone`; added `team` deprecation alias; added `DEFAULT_FEDERATED_CONFIG`. |
|
||||
| FED-M1-02 | done | Historical shipped-status record: authored a federated Compose overlay with PostgreSQL/pgvector, Valkey, volumes, and healthchecks. It is not a current startup, init, extension, schema, or migration procedure. | #460 | sonnet | feat/federation-m1-compose | FED-M1-01 | 5K | Shipped in PR #471 status only. KBN-101-02 retires its init authority; KBN-101-05 replaces deployment rendering; KBN-101-07 SETUP is non-operative until activation. |
|
||||
| FED-M1-03 | done | Historical shipped-status record: add pgvector support to `packages/storage/src/adapters/postgres.ts`; no adapter changes for non-federated tiers. | #460 | sonnet | feat/federation-m1-pgvector | FED-M1-02 | 8K | Shipped in PR #472 status only. **KBN-101 supersedes this behavior:** it cannot authorize current runtime extension creation or any DDL; only the runner/external bootstrap contract may do so. |
|
||||
| FED-M1-04 | done | Implement `apps/gateway/src/bootstrap/tier-detector.ts`: reads config, asserts PG/Valkey/pgvector reachable for `federated`, fail-fast with actionable error message on failure. Unit tests for each failure mode. | #460 | sonnet | feat/federation-m1-detector | FED-M1-03 | 8K | Shipped in PR #473. 12 tests; 5s timeouts on probes; pgvector library/permission discrimination; rejects non-bullmq for federated. |
|
||||
| FED-M1-05 | done | Historical shipped-status record: prior tier migration implementation. | #460 | sonnet | feat/federation-m1-migrate | FED-M1-04 | 10K | Shipped in PR #474 status only. **KBN-101 supersedes this route:** it cannot authorize current credentials, target connection, or DDL. The future active route requires runner verification plus target URL-file and signed attestation-file binding. |
|
||||
| FED-M1-06 | done | Update `mosaic doctor`: report current tier, required services, actual health per service, pgvector presence, overall green/yellow/red. Machine-readable JSON output flag for CI use. | #460 | sonnet | feat/federation-m1-doctor | FED-M1-04 | 6K | Shipped in PR #475 as `mosaic gateway doctor`. Probes lifted to @mosaicstack/storage; structural TierConfig breaks dep cycle. |
|
||||
| FED-M1-07 | done | Integration test: gateway boots in `federated` tier with docker-compose `federated` profile; refuses to boot when PG unreachable (asserts fail-fast); pgvector extension query succeeds. | #460 | sonnet | feat/federation-m1-integration | FED-M1-04 | 8K | Shipped in PR #476. 3 test files, 4 tests, gated by FEDERATED_INTEGRATION=1; reserved-port helper avoids host collisions. |
|
||||
| FED-M1-08 | done | Integration test for migration script: seed a local PGlite with representative data (tasks, notes, users, teams), run migration, assert row counts + key samples equal on federated PG. | #460 | sonnet | feat/federation-m1-migrate-test | FED-M1-05 | 6K | Shipped in PR #477. Caught P0 in M1-05 (camelCase→snake_case) missed by mocked unit tests; fix in same PR. |
|
||||
| FED-M1-09 | done | Standalone regression: full agent-session E2E on existing `standalone` tier with a gateway built from this branch. Must pass without referencing any federation module. | #460 | sonnet | feat/federation-m1-regression | FED-M1-07 | 4K | Clean canary. 351 gateway tests + 85 storage unit tests + full pnpm test all green; only FEDERATED_INTEGRATION-gated tests skip. |
|
||||
| FED-M1-10 | done | Code review pass: security-focused on the migration script (data-at-rest during migration) + tier detector (error-message sensitivity leakage). Independent reviewer, not authors of tasks 01-09. | #460 | sonnet | feat/federation-m1-security-review | FED-M1-09 | 8K | 2 review rounds caught 7 issues: credential leak in pg/valkey/pgvector errors + redact-error util; missing advisory lock; SKIP_TABLES rationale. |
|
||||
| FED-M1-11 | done | Docs update: `docs/federation/` operator notes for tier setup; README blurb on federated tier; `docs/guides/` entry for migration. Do NOT touch runbook yet (deferred to FED-M7). | #460 | haiku | feat/federation-m1-docs | FED-M1-10 | 4K | Shipped: `docs/federation/SETUP.md` (119 lines), `docs/guides/migrate-tier.md` (147 lines), README Configuration blurb. |
|
||||
| FED-M1-12 | done | PR, CI green, merge to main, close #460. | #460 | sonnet | feat/federation-m1-close | FED-M1-11 | 3K | M1 closed. PRs #470-#480 merged across 11 tasks. Issue #460 closed; release tag `fed-v0.1.0-m1` published. |
|
||||
|
||||
**M1 total estimate:** ~74K tokens (over-budget vs 20K PRD estimate — explanation below)
|
||||
|
||||
**Why over-budget:** PRD's 20K estimate reflected implementation complexity only. The per-task breakdown includes tests, review, and docs as separate tasks per the delivery cycle, which catches the real cost. The final per-milestone budgets in MISSION-MANIFEST will be updated after M1 completes with actuals.
|
||||
|
||||
---
|
||||
|
||||
## Pre-M2 — Test deployment infrastructure (FED-M2-DEPLOY)
|
||||
|
||||
Goal: Two federated-tier gateways stood up on Portainer at `mos-test-1.woltje.com` and `mos-test-2.woltje.com` running the M1 release (`gateway:fed-v0.1.0-m1`). This is the test bed for M2 enrollment work and the M3 federation E2E harness. No federation logic exercised yet — pure infrastructure validation.
|
||||
|
||||
> **Why now:** M2 enrollment requires a real second gateway to test peer-add flows; standing the test hosts up before M2 code lands gives both code and deployment streams a fast feedback loop.
|
||||
|
||||
> **Parallelizable:** This workstream runs in parallel with the M2 code workstream (M2-01 → M2-13). They re-converge at M2-10 (E2E test).
|
||||
|
||||
> **Tracking issue:** #482.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | ------ | ------------------------------------- | ------------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| FED-M2-DEPLOY-01 | done | Verify `gateway:fed-v0.1.0-m1` image was published by `.woodpecker/publish.yml` on tag push; if not, investigate and remediate. Document image URI in deployment artifact. | #482 | sonnet | (verified inline, no PR) | — | 2K | Tag exists; digest `sha256:9b72e202a9eecc27d31920b87b475b9e96e483c0323acc57856be4b1355db1ec` captured for digest-pinned deploys. |
|
||||
| FED-M2-DEPLOY-02 | done | Author Portainer git-stack compose file `deploy/portainer/federated-test.stack.yml` (gateway + PG-pgvector + Valkey, env-driven). Use immutable tag, not `latest`. | #482 | sonnet | feat/federation-deploy-stack-template | DEPLOY-01 | 5K | Shipped in PR #485. Digest-pinned. Env: STACK_NAME, HOST_FQDN, POSTGRES_PASSWORD, BETTER_AUTH_SECRET, BETTER_AUTH_URL. |
|
||||
| FED-M2-DEPLOY-IMG-FIX | in-progress | Gateway image runtime broken (ERR_MODULE_NOT_FOUND for `dotenv`); Dockerfile copies `.pnpm/` store but not `apps/gateway/node_modules` symlinks. Switch to `pnpm deploy` for self-contained runtime. | #482 | sonnet | (subagent in flight) | DEPLOY-02 | 4K | Subagent `a78a9ab0ddae91fbc` in flight. Triggers Kaniko rebuild on merge; capture new digest; bump stack template in follow-up PR before redeploy. |
|
||||
| FED-M2-DEPLOY-03 | blocked | Deploy stack to mos-test-1.woltje.com via `~/.config/mosaic/tools/portainer/`. Verify M1 acceptance: federated-tier boot succeeds; `mosaic gateway doctor --json` returns green; pgvector `vector(3)` round-trip works. | #482 | sonnet | feat/federation-deploy-test-1 | IMG-FIX | 3K | Stack created on Portainer endpoint 3 (Swarm `local`), but blocked on image fix. Container fails on boot until IMG-FIX merges + redeploy. |
|
||||
| FED-M2-DEPLOY-04 | blocked | Deploy stack to mos-test-2.woltje.com via Portainer wrapper. Same M1 acceptance probes as DEPLOY-03. | #482 | sonnet | feat/federation-deploy-test-2 | IMG-FIX | 3K | Same status as DEPLOY-03. Stack created; blocked on image fix. |
|
||||
| FED-M2-DEPLOY-05 | not-started | Document deployment in `docs/federation/TEST-INFRA.md`: hosts, image tags, secrets sourcing, redeploy procedure, teardown. Update MISSION-MANIFEST with deployment status. | #482 | haiku | feat/federation-deploy-docs | DEPLOY-03,04 | 3K | Operator-facing doc; mentions but does not duplicate `tools/portainer/README.md`. |
|
||||
|
||||
**Deploy workstream estimate:** ~16K tokens
|
||||
|
||||
---
|
||||
|
||||
## Milestone 2 — Step-CA + grant schema + admin CLI (FED-M2)
|
||||
|
||||
Goal: An admin can create a federation grant; counterparty enrolls; cert is signed by Step-CA with SAN OIDs for `grantId` + `subjectUserId`. No runtime federation traffic flows yet (that's M3).
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ---------------------------------- | ---------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| FED-M2-01 | done | DB migration: `federation_grants`, `federation_peers`, `federation_audit_log` tables + enum types (`grant_status`, `peer_state`). Drizzle schema + migration generation; migration tests. | #461 | sonnet | feat/federation-m2-schema | — | 5K | Shipped in PR #486. DESC indexes + reserved cols added after first review; migration tests green. |
|
||||
| FED-M2-02 | done | Add Step-CA sidecar to `docker-compose.federated.yml`: official `smallstep/step-ca` image, persistent CA volume, JWK provisioner config baked into init script. | #461 | sonnet | feat/federation-m2-stepca | DEPLOY-02 | 4K | Shipped in PR #494. Profile-gated under `federated`; CA password from secret; dev compose uses dev-only password file. |
|
||||
| FED-M2-03 | done | Scope JSON schema + validator: `resources` allowlist, `excluded_resources`, `include_teams`, `include_personal`, `max_rows_per_query`. Vitest unit tests for valid + invalid scopes. | #461 | sonnet | feat/federation-m2-scope-schema | — | 4K | Shipped in PR #496 (bundled with grants service). Validator independent of CA; reusable from grant CRUD + M3 scope enforcement. |
|
||||
| FED-M2-04 | done | `apps/gateway/src/federation/ca.service.ts`: Step-CA client (CSR submission, OID-bearing cert retrieval). Mocked + integration tests against real Step-CA container. | #461 | sonnet | feat/federation-m2-ca-service | M2-02 | 6K | Shipped in PR #494. SAN OIDs 1.3.6.1.4.1.99999.1 (grantId) + 1.3.6.1.4.1.99999.2 (subjectUserId); integration test asserts both OIDs present in issued cert. |
|
||||
| FED-M2-05 | done | Sealed storage for `client_key_pem` reusing existing `provider_credentials` sealing key. Tests prove DB-at-rest is ciphertext, not PEM. Key rotation path documented (deferred impl). | #461 | sonnet | feat/federation-m2-key-sealing | M2-01 | 5K | Shipped in PR #495. Crypto seam isolated; tests confirm ciphertext-at-rest; key rotation deferred to M6. |
|
||||
| FED-M2-06 | done | `grants.service.ts`: CRUD + status transitions (`pending` → `active` → `revoked`); integrates M2-03 (scope) + M2-05 (sealing). Unit tests cover all transitions including invalid ones. | #461 | sonnet | feat/federation-m2-grants-service | M2-03, M2-05 | 6K | Shipped in PR #496. All status transitions covered; invalid transition tests green; revocation handler deferred to M6. |
|
||||
| FED-M2-07 | done | `enrollment.controller.ts`: short-lived single-use token endpoint; CSR signing; updates grant `pending` → `active`; emits enrollment audit (table-only write, M4 tightens). | #461 | sonnet | feat/federation-m2-enrollment | M2-04, M2-06 | 6K | Shipped in PR #497. Tokens single-use with 410 on replay; TTL 15min; rate-limited at request layer. |
|
||||
| FED-M2-08 | done | Admin CLI: `mosaic federation grant create/list/show` + `peer add/list`. Integration with grants.service (no API duplication). Help output + machine-readable JSON option. | #461 | sonnet | feat/federation-m2-cli | M2-06, M2-07 | 7K | Shipped in PR #498. `peer add <enrollment-url>` client-side flow; JSON output flag; admin REST controller co-shipped. |
|
||||
| FED-M2-09 | done | Integration tests covering MILESTONES.md M2 acceptance tests #1, #2, #3, #5, #7, #8 (single-gateway suite). Real Step-CA container; vitest profile gated by `FEDERATED_INTEGRATION=1`. | #461 | sonnet | feat/federation-m2-integration | M2-08 | 8K | Shipped in PR #499. All 6 acceptance tests green; gated by FEDERATED_INTEGRATION=1. |
|
||||
| FED-M2-10 | done | E2E test against deployed mos-test-1 + mos-test-2 (or local two-gateway docker-compose if Portainer not ready): MILESTONES test #6 `peer add` yields `active` peer record with valid cert + key. | #461 | sonnet | feat/federation-m2-e2e | M2-08, DEPLOY-04 | 6K | Shipped in PR #500. Local two-gateway docker-compose path used; `peer add` yields active peer with valid cert + sealed key. |
|
||||
| FED-M2-11 | done | Independent security review (sonnet, not author of M2-04/05/06/07): focus on single-use token replay, sealing leak surfaces, OID match enforcement, scope schema bypass paths. | #461 | sonnet | feat/federation-m2-security-review | M2-10 | 8K | Shipped in PR #501. Two-round review; enrollment-token replay, OID-spoofing CSR, and key leak in error messages all verified and hardened. |
|
||||
| FED-M2-12 | done | Docs update: `docs/federation/SETUP.md` Step-CA section; new `docs/federation/ADMIN-CLI.md` with grant/peer commands; scope schema reference; OID registration note. Runbook still M7-deferred. | #461 | haiku | feat/federation-m2-docs | M2-11 | 4K | Shipped in PR #502. SETUP.md CA bootstrap section added; ADMIN-CLI.md created; scope schema reference and OID note included. |
|
||||
| FED-M2-13 | done | PR aggregate close, CI green, merge to main, close #461. Release tag `fed-v0.2.0-m2`. Mark deploy stream complete. Update mission manifest M2 row. | #461 | sonnet | chore/federation-m2-close | M2-12 | 3K | Release tag `fed-v0.2.0-m2` created; issue #461 closed; all M2 PRs #494–#502 merged to main. |
|
||||
|
||||
**M2 code workstream estimate:** ~72K tokens (vs MILESTONES.md 30K — same over-budget pattern as M1, where per-task breakdown including tests/review/docs catches the real cost).
|
||||
|
||||
**Deploy + code combined:** ~88K tokens.
|
||||
|
||||
## Milestone 3 — mTLS handshake + list/get + scope enforcement (FED-M3)
|
||||
|
||||
Goal: Two federated gateways exchange real data over mTLS. Inbound requests pass through cert validation → grant lookup → scope enforcement → native RBAC → response. `list`, `get`, and `capabilities` verbs land. The federation E2E harness (`tools/federation-harness/`) is the new permanent test bed for M3+ and is gated on every milestone going forward.
|
||||
|
||||
> **Critical trust boundary.** Every 401/403 path needs a test. Code review is non-negotiable; M3-12 budgets two review rounds.
|
||||
>
|
||||
> **Tracking issue:** #462.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| --------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | ------ | ------------------------------------ | --------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| FED-M3-01 | done | `packages/types/src/federation/` — request/response DTOs for `list`, `get`, `capabilities` verbs. Wire-format zod schemas + inferred TS types. Includes `FederationRequest`, `FederationListResponse<T>`, `FederationGetResponse<T>`, `FederationCapabilitiesResponse`, error envelope, `_source` tag. | #462 | sonnet | feat/federation-m3-types | — | 4K | Reusable from gateway server + client + harness. Pure types — no I/O, no NestJS. |
|
||||
| FED-M3-02 | done | `tools/federation-harness/` scaffold: `docker-compose.two-gateways.yml` (Server A + Server B + step-CA), `seed.ts` (provisions grants, peers, sample tasks/notes/credentials per scope variant), `harness.ts` helper (boots stack, returns typed clients). README documents harness use. | #462 | sonnet | feat/federation-m3-harness | DEPLOY-04 (soft) | 8K | Falls back to local docker-compose if `mos-test-1/-2` not yet redeployed (DEPLOY chain blocked on IMG-FIX). Permanent test infra used by M3+. |
|
||||
| FED-M3-03 | done | `apps/gateway/src/federation/server/federation-auth.guard.ts` (NestJS guard). Validates inbound client cert from Fastify TLS context, extracts `grantId` + `subjectUserId` from custom OIDs, loads grant from DB, asserts `status='active'`, attaches `FederationContext` to request. | #462 | sonnet | feat/federation-m3-auth-guard | M3-01 | 8K | Reuses OID parsing logic mirrored from `ca.service.ts` post-issuance verification. 401 on malformed/missing OIDs; 403 on revoked/expired/missing grant. |
|
||||
| FED-M3-04 | in-progress | `apps/gateway/src/federation/server/scope.service.ts`. Pipeline: (1) resource allowlist + excluded check, (2) native RBAC eval as `subjectUserId`, (3) scope filter intersection (`include_teams`, `include_personal`), (4) `max_rows_per_query` cap. Pure service — DB calls injected. | #462 | sonnet | feat/federation-m3-scope-service | M3-01 | 10K | Hardest correctness target in M3. Reuses `parseFederationScope` (M2-03). Returns either `{ allowed: true, filter }` or structured deny reason for audit. |
|
||||
| FED-M3-05 | in-progress | `apps/gateway/src/federation/server/verbs/list.controller.ts`. Wires AuthGuard → ScopeService → tasks/notes/memory query layer; applies row cap; tags rows with `_source`. Resource selector via path param. | #462 | sonnet | feat/federation-m3-verb-list | M3-03, M3-04 | 6K | Routes: `POST /api/federation/v1/list/:resource`. No body persistence. Audit write deferred to M4. |
|
||||
| FED-M3-06 | not-started | `apps/gateway/src/federation/server/verbs/get.controller.ts`. Single-resource fetch by id; same pipeline as list. 404 on not-found, 403 on RBAC/scope deny — both audited the same way. | #462 | sonnet | feat/federation-m3-verb-get | M3-03, M3-04 | 6K | `POST /api/federation/v1/get/:resource/:id`. Mirrors list controller patterns. |
|
||||
| FED-M3-07 | done | `apps/gateway/src/federation/server/verbs/capabilities.controller.ts`. Read-only enumeration: returns `{ resources, excluded_resources, max_rows_per_query, supported_verbs }` derived from grant scope. Always allowed for an active grant — no RBAC eval. | #462 | sonnet | feat/federation-m3-verb-capabilities | M3-03 | 4K | `GET /api/federation/v1/capabilities`. Smallest verb; useful sanity check that mTLS + auth guard work end-to-end. |
|
||||
| FED-M3-08 | done | `apps/gateway/src/federation/client/federation-client.service.ts`. Outbound mTLS dialer: picks `(certPem, sealed clientKey)` from `federation_peers`, unwraps key, builds undici Agent with mTLS, calls peer verb, parses typed response, wraps non-2xx into `FederationClientError`. | #462 | sonnet | feat/federation-m3-client | M3-01 | 8K | Independent of server stream — can land in parallel with M3-03/04. Cert/key cached per-peer; flushed by future M5/M6 logic. |
|
||||
| FED-M3-09 | done | `apps/gateway/src/federation/client/query-source.service.ts`. Accepts `source: "local" \| "federated:<host>" \| "all"` from gateway query layer; for `"all"` fans out to local + each peer in parallel; merges results; tags every row with `_source`. | #462 | sonnet | feat/federation-m3-query-source | M3-08 | 8K | Per-peer failure surfaces as `_partial: true` in response, not hard failure (sets up M5 offline UX). M5 adds caching + circuit breaker on top. |
|
||||
| FED-M3-10 | not-started | Integration tests for MILESTONES.md M3 acceptance #6 (malformed OIDs → 401; valid cert + revoked grant → 403) and #7 (`max_rows_per_query` cap). Real PG, mocked TLS context (Fastify req shim). | #462 | sonnet | feat/federation-m3-integration | M3-05, M3-06 | 8K | Vitest profile gated by `FEDERATED_INTEGRATION=1`. Single-gateway suite; no harness required. |
|
||||
| FED-M3-11 | not-started | E2E tests for MILESTONES.md M3 acceptance #1, #2, #3, #4, #5, #8, #9, #10 (8 cases). Uses harness from M3-02; two real gateways, real Step-CA, real mTLS. Each test asserts both happy-path response and audit/no-persist invariants. | #462 | sonnet | feat/federation-m3-e2e | M3-02, M3-04, M3-05, M3-06, M3-09 | 12K | Largest single task. Each acceptance gets its own `it(...)` for clear failure attribution. |
|
||||
| FED-M3-12 | not-started | Independent security review (sonnet, not author of M3-03/04/05/06/07/08/09): focus on cert-SAN spoofing, OID extraction edge cases, scope-bypass via filter manipulation, RBAC-bypass via subjectUser swap, response leakage when scope deny. | #462 | sonnet | feat/federation-m3-security-review | M3-11 | 10K | Two review rounds budgeted. PRD requires explicit test for every 401/403 path — review verifies coverage. |
|
||||
| FED-M3-13 | not-started | Docs update: `docs/federation/SETUP.md` mTLS handshake section, new `docs/federation/HARNESS.md` for federation-harness usage, OID reference table in SETUP.md, scope enforcement pipeline diagram. Runbook still M7-deferred. | #462 | haiku | feat/federation-m3-docs | M3-12 | 5K | One ASCII diagram for the auth-guard → scope → RBAC pipeline; helps future reviewers reason about denial paths. |
|
||||
| FED-M3-14 | not-started | PR aggregate close, CI green, merge to main, close #462. Release tag `fed-v0.3.0-m3`. Update mission manifest M3 row → done; M4 row → in-progress when work begins. | #462 | sonnet | chore/federation-m3-close | M3-13 | 3K | Same close pattern as M1-12 / M2-13. |
|
||||
|
||||
**M3 estimate:** ~100K tokens (vs MILESTONES.md 40K — same per-task breakdown pattern as M1/M2: tests, review, and docs split out from implementation cost). Largest milestone in the federation mission.
|
||||
|
||||
**Parallelization opportunities:**
|
||||
|
||||
- M3-08 (client) can land in parallel with M3-03/M3-04 (server pipeline) — they only share DTOs from M3-01.
|
||||
- M3-02 (harness) can land in parallel with everything except M3-11.
|
||||
- M3-05/M3-06/M3-07 (verbs) are independent of each other once M3-03/M3-04 land.
|
||||
|
||||
**Test bed fallback:** If `mos-test-1.woltje.com` / `mos-test-2.woltje.com` are still blocked on `FED-M2-DEPLOY-IMG-FIX` when M3-11 is ready to run, the harness's local `docker-compose.two-gateways.yml` is a sufficient stand-in. Production-host validation moves to M7 acceptance suite (PRD AC-12).
|
||||
|
||||
**Backlog sync — 2026-06-24 (orchestrator):** Status reconciled against `origin/main` (release 0.0.48). Landed on main: **FED-M3-01** (DTOs, PR #506), **FED-M3-02** (harness scaffold, PR #505), **FED-M3-03** (mTLS auth-guard, PR #509 — CRIT-1/2 + HIGH-1..4 remediated in-PR), **FED-M3-08** (outbound mTLS client, PR #508). With M3-01/03/08 merged, three cards became dependency-clear and were dispatched to the idle coder lane: **FED-M3-04** scope.service → coder0 (`feat/federation-m3-scope-service`); **FED-M3-09** query-source + **FED-M3-07** capabilities verb → coder1 (`feat/federation-m3-query-source` first). Reviewer warmed for the M3 trust-boundary PRs. Remaining blocked-by-DAG: M3-05/06 (await M3-04), M3-10 (await M3-05/06), M3-11 (await M3-09), M3-12→14 (tail). Deploy chain (DEPLOY-IMG-FIX → 03/04) still independent of M3 code — harness local docker-compose fallback covers M3-11.
|
||||
|
||||
**Backlog sync #2 — 2026-06-24 (orchestrator):** **FED-M3-09** (query-source) merged via PR #673 and **FED-M3-07** (capabilities) merged via PR #674 — both squash-merged on independent agent review-of-record + green CI (formal Gitea approve unavailable under the shared service account; merge is not gated by the self-approve guard). **FED-M3-05** (list verb) dispatched to coder1 (based on the M3-04 branch, rebase onto main once #672 lands). **FED-M3-04** (scope.service, PR #672) is in review-changes (one include_personal no-leak test outstanding). **DAG fix:** corrected `FED-M3-11` depends_on from `M3-02, M3-09` → `M3-02, M3-04, M3-05, M3-06, M3-09` — the E2E acceptance cases (#1–#5, #8–#10) exercise list/get over mTLS, so the server verbs + scope service are hard prerequisites; the original edge set omitted them and caused a premature M3-11 dispatch. Note: M3 read-path invariant for M3-11 is **no-persist + existing enrollment audit only** — read-verb audit-log writes are deferred to M4 (see M3-05/06 notes), so M3-11 must not assert read-audit-log entries.
|
||||
|
||||
## Milestone 4 — search + audit + rate limit (FED-M4)
|
||||
|
||||
_Deferred. Issue #463._
|
||||
|
||||
## Milestone 5 — cache + offline + OTEL (FED-M5)
|
||||
|
||||
_Deferred. Issue #464._
|
||||
|
||||
## Milestone 6 — revocation + auto-renewal + CRL (FED-M6)
|
||||
|
||||
_Deferred. Issue #465._
|
||||
|
||||
## Milestone 7 — multi-user hardening + acceptance suite (FED-M7)
|
||||
|
||||
_Deferred. Issue #466._
|
||||
|
||||
---
|
||||
|
||||
## Execution Notes
|
||||
|
||||
**Agent assignment rationale:**
|
||||
|
||||
- `codex` for most implementation tasks (OpenAI credit pool preferred for feature code)
|
||||
- `sonnet` for tests (pattern-based, moderate complexity), `doctor` work (cross-cutting), and independent code review
|
||||
- `haiku` for docs and the standalone regression canary (cheapest tier for mechanical/verification work)
|
||||
- No `opus` in M1 — save for cross-cutting architecture decisions if they surface later
|
||||
|
||||
**Branch strategy:** Each task gets its own feature branch off `main`. Tasks within a milestone merge in dependency order. Final aggregate PR (FED-M1-12) isn't a branch of its own — it's the merge of the last upstream task that closes the issue.
|
||||
|
||||
**Queue guard:** Every push and every merge in this mission must run `~/.config/mosaic/tools/git/ci-queue-wait.sh --purpose push|merge` per Mosaic hard gate #6.
|
||||
@@ -0,0 +1,78 @@
|
||||
# Fleet Configuration Management — Documentation IA Acceptance Checklist
|
||||
|
||||
**Issue:** #758 · **Scope:** M0 documentation gate for the local fleet declarative-configuration program.
|
||||
|
||||
This checklist is an acceptance contract for documentation and examples. It does not authorize
|
||||
schema, runtime, systemd, role, profile, or live-fleet changes. An item is complete only when its
|
||||
named artifact exists, is linked from the fleet documentation entry point, and its evidence is
|
||||
recorded in the M5 closure report and linked deferral evidence.
|
||||
|
||||
## M0 baseline acceptance
|
||||
|
||||
- [x] `docs/PRD.md` states the roster as desired-state SSOT; generated environment, systemd, tmux, and heartbeat artifacts as non-authoritative projections; and fail-closed handling of unsupported or quarantined legacy input.
|
||||
- [x] `docs/PRD.md` defines the required classes and authority boundary: `validator` certifies but does not merge; `merge-gate` remains sole approve-to-land/merge authority; `team-leader` capacity is lease-bounded; `interaction` is request/status only; instance names such as Tess and Ultron remain configurable.
|
||||
- [x] `docs/PRD.md` defines local lifecycle semantics for `enabled`, persisted desired state, and observed state, including stopped-state preservation through migration, apply, and reboot.
|
||||
- [x] `docs/PRD.md` defines the generated-env/local-override boundary, explicitly denies arbitrary command overrides in M1–M5, and requires key-name/hash-only quarantine diagnostics.
|
||||
- [x] `docs/PRD.md` identifies the M1–M5 local-tmux scope and excludes remote reconciliation, connector mutation, secret references, arbitrary commands/channels, gateway convergence, and UI configuration storage.
|
||||
- [x] `docs/TASKS.md` contains the complete M0–M5 one-card/one-PR dependency DAG for #758 with agent tier, branch, dependency, estimate, and evidence expectations.
|
||||
- [x] `docs/fleet/LEGACY-EXAMPLE-PROFILE-DISPOSITION-INVENTORY.md` classifies every current shipped fleet example, profile, and service preset before M1 implementation starts.
|
||||
|
||||
## Required documentation IA for M1–M5
|
||||
|
||||
| Path | Minimum content | Delivery gate |
|
||||
| ------------------------------------------------------- | ---------------------------------------------------------------------------- | ------------- |
|
||||
| `docs/fleet/README.md` | Fleet configuration entry point, desired-vs-observed decision tree, link map | M5 |
|
||||
| `docs/fleet/concepts/desired-vs-observed-state.md` | SSOT/projection model, drift, generation and ownership | M5 |
|
||||
| `docs/fleet/concepts/identity-class-runtime.md` | Stable name, display alias, class, runtime/provider/model separation | M5 |
|
||||
| `docs/fleet/concepts/role-authority-and-leases.md` | Required roles, validator/merge-gate separation, lease limits | M5 |
|
||||
| `docs/fleet/concepts/generated-env-launch-chain.md` | Generated/local files, precedence, quarantine and non-shell parsing | M5 |
|
||||
| `docs/fleet/reference/roster-v2.schema.json` | Executable v2 structural contract | M1 |
|
||||
| `docs/fleet/reference/roster-v2-fields.md` | Every field, default, constraint, compatibility behavior and examples | M1 |
|
||||
| `docs/fleet/reference/cli.md` | `config`, `agent`, lifecycle, plan/apply, JSON and exit-code contracts | M2–M3 |
|
||||
| `docs/fleet/reference/role-classes.md` | Canonical classes, aliases, authority matrix and instance-name rule | M1 |
|
||||
| `docs/fleet/reference/lifecycle-transitions.md` | Create/start/stop/restart/apply/reboot/rollback transition table | M3 |
|
||||
| `docs/fleet/reference/status-and-drift.md` | Desired/observed/managed state, orphans, revision mismatch, doctor output | M3 |
|
||||
| `docs/fleet/how-to/create-update-delete-agent.md` | Safe CRUD, expected generation, dry-run and rollback | M2 |
|
||||
| `docs/fleet/how-to/start-stop-restart.md` | Persisted versus one-shot lifecycle actions | M3 |
|
||||
| `docs/fleet/how-to/configure-tess-interaction.md` | Configurable interaction instance; no hardcoded identity | M5 |
|
||||
| `docs/fleet/how-to/configure-ultron-validator.md` | Configurable validator instance; no merge authority | M5 |
|
||||
| `docs/fleet/how-to/customize-roles.md` | Existing baseline + `roles.local` resolution and validation | M1 |
|
||||
| `docs/fleet/operations/reconcile-and-recover.md` | Plan/apply failure recovery, generation lock and canary rollout | M3 |
|
||||
| `docs/fleet/operations/env-quarantine.md` | Legacy-key inventory, private quarantine and redaction behavior | M2 |
|
||||
| `docs/fleet/operations/systemd-tmux-troubleshooting.md` | Socket ambiguity, ownership proof, systemd/tmux drift | M3 |
|
||||
| `docs/fleet/operations/backup-restore.md` | Roster/projection backup and rollback boundaries | M4 |
|
||||
| `docs/fleet/operations/upgrade-assets.md` | Source-vs-installed asset revision detection and safe refresh | M5 |
|
||||
| `docs/fleet/migration/v1-to-v2.md` | Normative field map, observed-state preservation and rollback | M4 |
|
||||
| `docs/fleet/migration/example-profile-disposition.md` | Final disposition of every shipped example/profile | M1–M4 |
|
||||
| `docs/fleet/migration/legacy-class-aliases.md` | Alias, unresolved-class, and retirement rules | M1 |
|
||||
|
||||
## PRD acceptance-criteria mapping
|
||||
|
||||
| PRD acceptance criterion | Owning card(s) | Required evidence |
|
||||
| ------------------------------------------------------------ | ---------------------------------- | --------------------------------------------------------------------------------------- |
|
||||
| `AC-FCM-01` schema, semantic validation, canonical rendering | FCM-M1-001, FCM-M1-002 | YAML/JSON positive/negative and schema/parser/resolver parity tests |
|
||||
| `AC-FCM-02` deterministic plan and no-mutation check | FCM-M3-001 | Stable JSON/exit-code and desired-versus-observed fixture tests |
|
||||
| `AC-FCM-03` safe generation-guarded CRUD | FCM-M2-002 | Create/update/delete idempotency, expected-generation, dry-run, and recovery tests |
|
||||
| `AC-FCM-04` generated/local boundary and quarantine | FCM-M2-001 | Launch-chain, shadow, injection, redaction, and forbidden-key tests |
|
||||
| `AC-FCM-05` lifecycle/reconcile/socket/drift safety | FCM-M3-001, FCM-M3-002 | Isolated systemd/tmux, stopped-state, orphan, socket, and rollback evidence |
|
||||
| `AC-FCM-06` v1 migration and example/profile disposition | FCM-M4-001, FCM-M4-002, FCM-M1-003 | Preview/canary/rollback fixture plus executable disposition inventory |
|
||||
| `AC-FCM-07` authority and lease boundaries | FCM-M1-002 | Role/authority/lease denial tests and resolved role contracts |
|
||||
| `AC-FCM-08` documentation and final release gate | FCM-M5-001, FCM-M5-002 | Checklist closure, link/example validation, reviews, certificate, and terminal-green CI |
|
||||
|
||||
## Cross-cutting evidence gates
|
||||
|
||||
- [x] Every retained or migrated YAML/JSON example, profile, and service preset validates through the same declared executable production parser/resolver path recorded by the disposition inventory; versioned v1 fixtures are not forced through the v2 compiler.
|
||||
- [x] Every retired example/profile/service preset has a replacement link and deprecation note; no unresolved legacy class or tool-policy alias remains silently shipped.
|
||||
- [x] Documentation examples contain no secret values, arbitrary command override, or product-hardcoded Tess/Ultron identity.
|
||||
- [x] CLI snippets distinguish local fleet desired-state commands from the separate gateway-backed mosaic agent catalog.
|
||||
- [x] Migration, quarantine, lifecycle, status, and troubleshooting documentation state that values of legacy sensitive keys are never printed.
|
||||
- [x] M5 documentation validation verifies required IA paths, local file and heading-fragment links, the canonical roster through the production compiler/resolver, and fenced/canonical-example safety checks.
|
||||
- [ ] FCM-M5-001 does not deterministically assert owner/evidence/deferral metadata for every checklist row. Closure and deferral reports provide human-reviewable evidence only; broader assertion coverage remains unclaimed.
|
||||
|
||||
## Held downstream gates
|
||||
|
||||
These unchecked items are intentionally outside FCM-M5-001 and are not authorized by this checklist:
|
||||
|
||||
- [ ] FCM-M4-002 executes and evidences live cutover, canary, stopped-state preservation, and rollback.
|
||||
- [ ] FCM-M5-002 completes independent exact-head review and issues the validator certificate.
|
||||
- [ ] The exact PR head reaches terminal-green CI after independent review.
|
||||
@@ -0,0 +1,81 @@
|
||||
# Fleet Launch Runbook
|
||||
|
||||
The local fleet roster is the sole writable desired-state authority for membership and launch policy.
|
||||
Generated environment files are rebuildable projections, not an operator-editable command surface.
|
||||
|
||||
## Launch chain
|
||||
|
||||
| Layer | Responsibility |
|
||||
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Roster | `fleet/roster.yaml` supplies the agent name, class, supported runtime, model, reasoning, tool policy, workdir, and tmux socket. |
|
||||
| Projection writer | Renders deterministic fleet/agents/<name>.env.generated from the roster. |
|
||||
| Optional local data | Reads a strict, data-only fleet/agents/<name>.env.local; it cannot shadow generated keys. |
|
||||
| systemd | Starts the launcher with env -i and fixed bootstrap data. It does not preload either environment file. |
|
||||
| session launcher | Validates generated and local data before it queries, creates, or stops an exact tmux session. |
|
||||
| runtime launch | Derives the fixed mosaic yolo <runtime> argument array from validated roster data, then seeds the runtime contract. |
|
||||
|
||||
The launcher never `source`s or `eval`s an environment file and never accepts an environment-supplied
|
||||
command. `MOSAIC_AGENT_COMMAND`, command/channel overrides, unknown keys, generated-key shadowing,
|
||||
secret-like key names, duplicate keys, comments, quoted/export syntax, and unsafe values are rejected.
|
||||
|
||||
## Generated and local files
|
||||
|
||||
<name>.env.generated is complete, deterministic, and written only by Mosaic. Its ordered keys are:
|
||||
|
||||
```dotenv
|
||||
MOSAIC_AGENT_NAME=<roster name>
|
||||
MOSAIC_AGENT_CLASS=<roster class>
|
||||
MOSAIC_AGENT_RUNTIME=<roster runtime>
|
||||
MOSAIC_AGENT_MODEL=<roster model hint>
|
||||
MOSAIC_AGENT_REASONING=<roster reasoning>
|
||||
MOSAIC_AGENT_TOOL_POLICY=<roster tool policy>
|
||||
MOSAIC_AGENT_WORKDIR=<absolute roster work directory>
|
||||
MOSAIC_TMUX_SOCKET=<roster socket or empty>
|
||||
```
|
||||
|
||||
The generated launch contract supports `claude`, `codex`, `opencode`, and `pi`. mosaic fleet add
|
||||
rejects another runtime before it writes the roster or modifies generated, local, or quarantine state.
|
||||
The legacy dogfood stub remains an observability-only canary on its separate `mosaic-factory` socket;
|
||||
it has no generated-launch adapter and cannot be added through this path.
|
||||
|
||||
<name>.env.local is optional and may contain only non-secret machine data:
|
||||
|
||||
- `MOSAIC_RUNTIME_BIN`
|
||||
- `MOSAIC_HEARTBEAT_RUN_DIR`
|
||||
- `MOSAIC_HEARTBEAT_INTERVAL`
|
||||
- `MOSAIC_CLAUDE_JSON`
|
||||
- `CLAUDE_CONFIG_DIR`
|
||||
|
||||
Paths must be safe absolute paths and the heartbeat interval must be a positive integer. Projection,
|
||||
local, and quarantine files must be private regular files; the managed directories must be real,
|
||||
private, non-symlink paths. Violations fail closed before tmux interaction.
|
||||
|
||||
## Legacy input and diagnostics
|
||||
|
||||
A legacy <name>.env is input only during projection generation. Roster-owned keys are regenerated;
|
||||
valid allowed local data can move to `.env.local`; invalid legacy input is privately retained at
|
||||
<name>.env.quarantine. Neither legacy nor quarantine files are launch authority.
|
||||
|
||||
Diagnostics expose only rule code, key name, and a SHA-256 content hash. They do not reveal command
|
||||
text, credentials, or other values.
|
||||
|
||||
## Launch and stop behavior
|
||||
|
||||
The launcher obtains the agent's socket only from the validated generated projection. It creates or
|
||||
checks the exact =<agent-name> tmux target; it never uses an ambient socket or fuzzy session match.
|
||||
The same strict parser runs before exact-stop behavior. A fresh native Pi heartbeat remains authoritative;
|
||||
the shell sidecar only provides fallback state when the native marker is stale or absent.
|
||||
|
||||
mosaic agent comms-block <exact-member> can inspect that exact roster member's resolved Fleet-Comms
|
||||
block. It is a read-only inspection tool and fails loudly for an unknown exact member or missing roster.
|
||||
On Linux, the installed roster, TOOLS contract, and executable helper are opened through a held
|
||||
descriptor chain rooted at `/`; every managed path component uses no-follow traversal, and content plus
|
||||
execute validation stay bound to the same opened file. Systems without Linux `/proc/self/fd` support
|
||||
fail closed rather than falling back to pathname revalidation.
|
||||
|
||||
## Current M2 boundary
|
||||
|
||||
FCM-M2-001 supplies generated/local parsing, validation, projection, quarantine, and launch-boundary
|
||||
evidence only. It does not authorize roster CRUD expansion, reconciliation, lifecycle changes, remote
|
||||
or connector mutation, site canaries, or migration. M3 must establish the local reconcile/lifecycle
|
||||
path; M4 separately provides migration preview, canary, and rollback gates.
|
||||
@@ -0,0 +1,57 @@
|
||||
# Fleet Configuration Management — Legacy Example, Profile, and Service Disposition Inventory
|
||||
|
||||
**Issue:** #758 · **Baseline:** `origin/main` `49e8a541` · **Status:** M0 inventory; no source
|
||||
examples or profiles are changed by this document.
|
||||
|
||||
The v2 compiler may not silently accept an unresolved class. Before M1 exits, every shipped file
|
||||
below must be either migrated and executable, retained as an explicitly versioned v1 fixture, or
|
||||
retired with a replacement/deprecation note. Class resolution must use the existing
|
||||
profile/persona/provision baseline-plus-`roles.local` resolver; this inventory does not create a
|
||||
parallel resolver. The current executable implementation and per-artifact outcomes are recorded in
|
||||
[the disposition evidence](./migration/example-profile-disposition.md).
|
||||
|
||||
## Examples
|
||||
|
||||
| Shipped file | Current class evidence | M0 disposition decision | Required M1/M4 evidence |
|
||||
| ---------------------------------------------------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
|
||||
| `framework/fleet/examples/coding.yaml` | `orchestrator`, `enhancer`, `implementer`, `reviewer` | Migrate: implementer → code, reviewer → review; retain orchestration/enhancer intent | v2 fixture validates; role aliases and authority matrix tested |
|
||||
| `framework/fleet/examples/general.yaml` | `orchestrator`, `enhancer`, `worker` | Migrate only after operator chooses a concrete canonical role for `worker`; no implicit conversion | Explicit replacement class, or versioned v1 fixture/retirement note |
|
||||
| `framework/fleet/examples/hybrid.yaml` | `orchestrator`, `enhancer`, `implementer`, `researcher`, `reviewer` | Migrate aliases; resolve `researcher` through existing role resolver or retain/version | Shared resolver validation; no ad-hoc class scanner |
|
||||
| `framework/fleet/examples/local-canary.yaml` | `orchestrator`, `implementer`, `reviewer` | Migrate aliases; preserve its local-tmux canary purpose | v2 fixture validates and preserves safe stopped/running behavior |
|
||||
| `framework/fleet/examples/minimal.yaml` | `canary` | Retire or version as v1 unless an existing canonical role contract is selected deliberately | Replacement link/deprecation note or CI-valid v1 fixture |
|
||||
| `framework/fleet/examples/operator-interaction.yaml` | `operator-interaction` | Migrate alias to `interaction`; preserve instance/display name as configuration, not schema identity | v2 interaction fixture validates; no Tess literal is required |
|
||||
| `framework/fleet/examples/research.yaml` | `orchestrator`, `enhancer`, `researcher`, `analyst` | Resolve `researcher`/`analyst` through baseline + `roles.local`, or version/retire | Resolver evidence and explicit disposition for each unresolved class |
|
||||
|
||||
## Profiles
|
||||
|
||||
| Shipped file | Current class evidence | M0 disposition decision | Required M1/M4 evidence |
|
||||
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------- |
|
||||
| `framework/fleet/profiles/business.yaml` | `ceo`, `coo`, `cfo`, `product-manager`, `marketing-lead`, `sales-lead`, `operations-manager`, `customer-success-manager`, `code`, `review` | Retain only if every class resolves through the existing role library/`roles.local`; otherwise version/retire rather than weakening validation | Shared resolver CI result for every class; documented role source or replacement |
|
||||
| `framework/fleet/profiles/marketing.yaml` | `marketing-lead`, `content-strategist`, `copywriter`, `seo-specialist`, `social-media-manager`, `brand-strategist`, `growth-marketer`, `ux-designer` | Same resolver-or-version/retire rule | Per-class resolver CI result and replacement/deprecation record if unresolved |
|
||||
| `framework/fleet/profiles/personal-assistant.yaml` | `personal-assistant`, `executive-assistant`, `scheduler`, `inbox-manager`, `researcher` | Same resolver-or-version/retire rule | Per-class resolver CI result; do not infer `interaction` equivalence |
|
||||
| `framework/fleet/profiles/research.yaml` | `lead-researcher`, `researcher`, `data-analyst`, `data-scientist`, `market-analyst`, `documentation`, `review` | Same resolver-or-version/retire rule | Per-class resolver CI result and explicit compatibility posture |
|
||||
| `framework/fleet/profiles/software-delivery.yaml` | `orchestrator`, `board`, `planner`, `decomposition`, `code`, `review`, `security-review`, `site-tester`, `documentation`, `merge-gate`, `rebase`, `operator`, `session-review`, `enhancer` | Retain as the governance reference; add `validator`, `team-leader`, and `interaction` only through approved role/profile work, not silent substitution | CI validates all current classes; separate fixture proves required M1 authority seats |
|
||||
|
||||
## Service presets
|
||||
|
||||
| Shipped file | Current policy evidence | M0 disposition decision | Required M1/M4 evidence |
|
||||
| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `framework/fleet/services/operator-interaction.yaml` | Generic policy only: runtime: pi, model: openai/gpt-5.6-sol, reasoning: high, tool_policy: operator-interaction; provisioning supplies the agent name as data | Retain as a generic service policy, not a Tess identity. Migrate tool_policy: operator-interaction only through the approved interaction tool-policy alias/semantic resolver; do not infer a class or machine name from this file. | Service-policy fixture validates runtime/model/reasoning and alias behavior; generic provisioning proves a configured interaction instance is supplied without a hardcoded Tess name. |
|
||||
|
||||
## Required disposition controls
|
||||
|
||||
1. **No silent aliasing:** only implementer → code, reviewer → review, and
|
||||
operator-interaction → interaction are approved deterministic aliases in this M0 baseline.
|
||||
`worker`, `analyst`, `canary`, and domain-specific classes require resolver evidence or an
|
||||
explicit version/retirement decision.
|
||||
2. **No identity hardcoding:** Tess and Ultron are optional instance/display names. An example/profile
|
||||
may demonstrate the capability but must not make a product name a required class or machine ID.
|
||||
3. **No lifecycle inference from an example:** examples describe desired configuration only; migration
|
||||
of an installed v1 roster separately preserves observed stopped/running state.
|
||||
4. **No secret or command migration:** examples/profiles must not introduce credential values or
|
||||
`MOSAIC_AGENT_COMMAND`; those legacy keys are M2 quarantine inputs, never v2 authoring fields.
|
||||
5. **Service presets are included:** service policies are inventoried alongside examples/profiles.
|
||||
They may express launch/tool policy, but do not create a class, a canonical agent identity, or a
|
||||
second validation path.
|
||||
6. **Evidence is executable:** M1/M4 CI must enumerate these exact files, validate retained/migrated
|
||||
inputs through the shared resolver, and fail if a file lacks its documented disposition.
|
||||
@@ -0,0 +1,79 @@
|
||||
# Mosaic Fleet — NORTH STAR
|
||||
|
||||
> **Generated file — do not edit by hand.**
|
||||
> Projected deterministically from [`NORTH_STAR.yaml`](./NORTH_STAR.yaml) by the pure
|
||||
> generator in `packages/mosaic/src/commands/fleet.ts` (`renderNorthStarMarkdown`).
|
||||
> Edit the YAML, then regenerate. Self-contained Mosaic — no Hermes dependency.
|
||||
|
||||
## Mission
|
||||
|
||||
A self-driving Mosaic system that 24/7 unattended converts a machine-readable goal set into merged, CI-green, budget-bounded change — looping plan→backlog→assign→execute→verify→merge→reassess — on Mosaic's OWN native backlog/dispatch engine. Mosaic is general-purpose: the user declares the system type they want (software delivery, personal assistant, research, business/operations, …) and the orchestrator provisions the matching persona roster and structure; the delivery fleet is one profile among many.
|
||||
|
||||
## Substrate
|
||||
|
||||
The Mosaic Backlog is the backlog of record + dispatch engine, built on Mosaic's native Postgres storage service (@mosaicstack/db drizzle; PGlite-embedded by default, full Postgres by config). NOT Hermes.
|
||||
|
||||
## Standing objectives
|
||||
|
||||
- **NS-1** — Single machine-readable source (this file) drives planning; prose docs are projections.
|
||||
- **NS-2** — Every backlog item is an independently-shippable unit with stable id, priority, depends_on DAG, represented as a Mosaic Backlog card; spend tracked as advisory projection.
|
||||
- **NS-3** — The supervisor guarantees movement: no idle agent while ready dependency-satisfied work exists; no empty backlog without a replan request; assignment via Mosaic native dispatch/claim.
|
||||
- **NS-4** — Exactly one merge-gate approver; nothing reaches main except via pr-merge.sh after pr-ci-wait.sh success; Gitea branch protection is the backstop.
|
||||
- **NS-5** — Every unit bounded by wall-clock TTL on its claim; token caps enforced only where a real meter exists, else advisory.
|
||||
- **NS-6** — Context cleared between tasks for ephemeral runners (reset_between_tasks); persona+mission re-injected per task.
|
||||
- **NS-7** — Meta-loop (session-review + enhancer) continuously proposes small fleet-improvement PRs.
|
||||
- **NS-8** — Single operator-flippable PAUSE kill-switch (fleet/run/PAUSED) honored before every dispatch and every merge.
|
||||
- **NS-9** — Mosaic is a general-purpose multi-agent system: the user declares the SYSTEM TYPE to run (e.g. software delivery, personal assistant, research, business/operations) and the orchestrator provisions the matching persona roster and org structure from a cross-domain baseline persona library; the delivery/coding fleet is one profile among many.
|
||||
|
||||
## Success criteria
|
||||
|
||||
- **AC-NS-1** — The supervisor keeps a two-agent floor (1 orchestrator + >=1 enhancer) healthy across reboot.
|
||||
- **AC-NS-2** — A goal added to this YAML is decomposed to cards and either merged or escalated, with no human in the loop.
|
||||
- **AC-NS-3** — No PR merges with failure/error/no-status/timeout CI, and none bypass pr-merge.sh.
|
||||
- **AC-NS-4** — TTL is enforced on claims; token caps remain advisory until a real meter exists.
|
||||
- **AC-NS-5** — Flipping fleet/run/PAUSED halts dispatch and merges within one tick.
|
||||
- **AC-NS-6** — A user can declare a system type and the fleet provisions the matching persona roster + topology from the baseline library, with no code change.
|
||||
- **AC-NS-7** — A user-customized persona (edited or added via the orchestrator) survives mosaic update: baseline reseed never clobbers user overrides.
|
||||
|
||||
## Workstreams
|
||||
|
||||
| id | title |
|
||||
| --- | ----------------------------------------------------------------------------------------------------------- |
|
||||
| A | Substrate — Mosaic Backlog on native Postgres storage service |
|
||||
| B | Supervisor — movement guarantee, two-agent floor, dispatch/claim |
|
||||
| C | Planner — goal decomposition into independently-shippable cards |
|
||||
| D | Merge-gate — single approver, pr-merge.sh after CI wait |
|
||||
| E | Meta-loop — session-review + enhancer improvement PRs |
|
||||
| F | Safety-rails — TTL claims, advisory spend, PAUSE kill-switch |
|
||||
| H | Personas & system profiles — cross-domain library, system-type provisioning, update-surviving customization |
|
||||
|
||||
## Goals (backlog projection)
|
||||
|
||||
| id | title | phase | priority | depends_on |
|
||||
| --- | ------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | ----------- | ---------- |
|
||||
| A1 | Machine-readable NORTH_STAR.yaml + Markdown projection | 1 | must-have | — |
|
||||
| A2 | Mosaic Backlog schema + storage-service card store (drizzle/PGlite) | 1 | must-have | A1 |
|
||||
| A3a | Card lifecycle — create/claim/release with stable ids + depends_on DAG | 1 | must-have | A2 |
|
||||
| A3b | TTL-bounded claim enforcement (wall-clock) on cards | 1 | must-have | A3a |
|
||||
| A4 | Advisory spend projection per card (degrades to TTL, no real meter) | 1 | should-have | A3a |
|
||||
| B1 | Supervisor tick — readiness scan, two-agent-floor health check | 2 | must-have | A3a |
|
||||
| B2 | Native dispatch/claim — assign ready dependency-satisfied work | 2 | must-have | A3b, B1 |
|
||||
| B3a | Planner decompose — goal added to YAML → cards | 2 | must-have | A2, B1 |
|
||||
| B3b | Replan request on empty backlog; escalate on no-decompose | 2 | should-have | B3a |
|
||||
| G1 | PAUSE kill-switch + merge-gate honored before dispatch and merge | 2 | must-have | B2 |
|
||||
| H1 | Cross-domain baseline persona library (exec, marketing, ops, research, assistant + engineering roles) | 1 | must-have | A1 |
|
||||
| H2 | System-type profiles — declarative mapping of system type to persona roster + topology | 2 | must-have | H1 |
|
||||
| H3 | System-type provisioning — user declares type; orchestrator instantiates the matching roster + structure | 2 | must-have | H2 |
|
||||
| H4 | Update-surviving persona customization — ad-hoc edits/additions persisted in a PRESERVE-protected override layer (baseline merged with overrides) | 2 | must-have | H1 |
|
||||
|
||||
## Assumptions (vetoable)
|
||||
|
||||
- **ASM-1** (vetoable) — The Mosaic Backlog on the native Postgres storage service is the backlog of record.
|
||||
- **ASM-2** (vetoable) — Claude gate roles have no native busy status, so readiness = pane-idle + heartbeat.
|
||||
- **ASM-3** (vetoable) — Two-agent floor = 1 orchestrator + >=1 enhancer.
|
||||
- **ASM-4** (vetoable) — Baseline personas ship in framework/fleet/roles/ (reseeded on update); user overrides live in a separate PRESERVE_PATHS-protected layer and win on merge.
|
||||
|
||||
## Spend
|
||||
|
||||
- **advisory:** true
|
||||
- No per-task token meter yet; budgets degrade to TTL. Spend is tracked only as an advisory projection alongside each card.
|
||||
@@ -0,0 +1,215 @@
|
||||
# Mosaic Fleet — NORTH_STAR (machine-readable source of truth)
|
||||
#
|
||||
# This file is the single machine-readable source of truth for fleet planning.
|
||||
# Prose docs (including NORTH_STAR.md) are deterministic PROJECTIONS of this file.
|
||||
# Regenerate the Markdown projection with the pure generator in
|
||||
# packages/mosaic/src/commands/fleet.ts (renderNorthStarMarkdown). Edit the YAML,
|
||||
# never the .md.
|
||||
#
|
||||
# Self-contained Mosaic. NO Hermes runtime dependency. The backlog of record is
|
||||
# the Mosaic Backlog on Mosaic's OWN native Postgres storage service.
|
||||
|
||||
version: 1
|
||||
|
||||
mission: >-
|
||||
A self-driving Mosaic system that 24/7 unattended converts a machine-readable
|
||||
goal set into merged, CI-green, budget-bounded change — looping
|
||||
plan→backlog→assign→execute→verify→merge→reassess — on Mosaic's OWN native
|
||||
backlog/dispatch engine. Mosaic is general-purpose: the user declares the
|
||||
system type they want (software delivery, personal assistant, research,
|
||||
business/operations, …) and the orchestrator provisions the matching persona
|
||||
roster and structure; the delivery fleet is one profile among many.
|
||||
|
||||
substrate:
|
||||
note: >-
|
||||
The Mosaic Backlog is the backlog of record + dispatch engine, built on
|
||||
Mosaic's native Postgres storage service (@mosaicstack/db drizzle;
|
||||
PGlite-embedded by default, full Postgres by config). NOT Hermes.
|
||||
|
||||
standing_objectives:
|
||||
- id: NS-1
|
||||
text: >-
|
||||
Single machine-readable source (this file) drives planning; prose docs are
|
||||
projections.
|
||||
- id: NS-2
|
||||
text: >-
|
||||
Every backlog item is an independently-shippable unit with stable id,
|
||||
priority, depends_on DAG, represented as a Mosaic Backlog card; spend
|
||||
tracked as advisory projection.
|
||||
- id: NS-3
|
||||
text: >-
|
||||
The supervisor guarantees movement: no idle agent while ready
|
||||
dependency-satisfied work exists; no empty backlog without a replan
|
||||
request; assignment via Mosaic native dispatch/claim.
|
||||
- id: NS-4
|
||||
text: >-
|
||||
Exactly one merge-gate approver; nothing reaches main except via
|
||||
pr-merge.sh after pr-ci-wait.sh success; Gitea branch protection is the
|
||||
backstop.
|
||||
- id: NS-5
|
||||
text: >-
|
||||
Every unit bounded by wall-clock TTL on its claim; token caps enforced
|
||||
only where a real meter exists, else advisory.
|
||||
- id: NS-6
|
||||
text: >-
|
||||
Context cleared between tasks for ephemeral runners
|
||||
(reset_between_tasks); persona+mission re-injected per task.
|
||||
- id: NS-7
|
||||
text: >-
|
||||
Meta-loop (session-review + enhancer) continuously proposes small
|
||||
fleet-improvement PRs.
|
||||
- id: NS-8
|
||||
text: >-
|
||||
Single operator-flippable PAUSE kill-switch (fleet/run/PAUSED) honored
|
||||
before every dispatch and every merge.
|
||||
- id: NS-9
|
||||
text: >-
|
||||
Mosaic is a general-purpose multi-agent system: the user declares the
|
||||
SYSTEM TYPE to run (e.g. software delivery, personal assistant, research,
|
||||
business/operations) and the orchestrator provisions the matching persona
|
||||
roster and org structure from a cross-domain baseline persona library; the
|
||||
delivery/coding fleet is one profile among many.
|
||||
|
||||
success_criteria:
|
||||
- id: AC-NS-1
|
||||
text: >-
|
||||
The supervisor keeps a two-agent floor (1 orchestrator + >=1 enhancer)
|
||||
healthy across reboot.
|
||||
- id: AC-NS-2
|
||||
text: >-
|
||||
A goal added to this YAML is decomposed to cards and either merged or
|
||||
escalated, with no human in the loop.
|
||||
- id: AC-NS-3
|
||||
text: >-
|
||||
No PR merges with failure/error/no-status/timeout CI, and none bypass
|
||||
pr-merge.sh.
|
||||
- id: AC-NS-4
|
||||
text: >-
|
||||
TTL is enforced on claims; token caps remain advisory until a real meter
|
||||
exists.
|
||||
- id: AC-NS-5
|
||||
text: >-
|
||||
Flipping fleet/run/PAUSED halts dispatch and merges within one tick.
|
||||
- id: AC-NS-6
|
||||
text: >-
|
||||
A user can declare a system type and the fleet provisions the matching
|
||||
persona roster + topology from the baseline library, with no code change.
|
||||
- id: AC-NS-7
|
||||
text: >-
|
||||
A user-customized persona (edited or added via the orchestrator) survives
|
||||
mosaic update: baseline reseed never clobbers user overrides.
|
||||
|
||||
workstreams:
|
||||
- id: A
|
||||
title: Substrate — Mosaic Backlog on native Postgres storage service
|
||||
- id: B
|
||||
title: Supervisor — movement guarantee, two-agent floor, dispatch/claim
|
||||
- id: C
|
||||
title: Planner — goal decomposition into independently-shippable cards
|
||||
- id: D
|
||||
title: Merge-gate — single approver, pr-merge.sh after CI wait
|
||||
- id: E
|
||||
title: Meta-loop — session-review + enhancer improvement PRs
|
||||
- id: F
|
||||
title: Safety-rails — TTL claims, advisory spend, PAUSE kill-switch
|
||||
- id: H
|
||||
title: Personas & system profiles — cross-domain library, system-type provisioning, update-surviving customization
|
||||
|
||||
goals:
|
||||
- id: A1
|
||||
title: Machine-readable NORTH_STAR.yaml + Markdown projection
|
||||
phase: 1
|
||||
priority: must-have
|
||||
depends_on: []
|
||||
- id: A2
|
||||
title: Mosaic Backlog schema + storage-service card store (drizzle/PGlite)
|
||||
phase: 1
|
||||
priority: must-have
|
||||
depends_on: [A1]
|
||||
- id: A3a
|
||||
title: Card lifecycle — create/claim/release with stable ids + depends_on DAG
|
||||
phase: 1
|
||||
priority: must-have
|
||||
depends_on: [A2]
|
||||
- id: A3b
|
||||
title: TTL-bounded claim enforcement (wall-clock) on cards
|
||||
phase: 1
|
||||
priority: must-have
|
||||
depends_on: [A3a]
|
||||
- id: A4
|
||||
title: Advisory spend projection per card (degrades to TTL, no real meter)
|
||||
phase: 1
|
||||
priority: should-have
|
||||
depends_on: [A3a]
|
||||
- id: B1
|
||||
title: Supervisor tick — readiness scan, two-agent-floor health check
|
||||
phase: 2
|
||||
priority: must-have
|
||||
depends_on: [A3a]
|
||||
- id: B2
|
||||
title: Native dispatch/claim — assign ready dependency-satisfied work
|
||||
phase: 2
|
||||
priority: must-have
|
||||
depends_on: [A3b, B1]
|
||||
- id: B3a
|
||||
title: Planner decompose — goal added to YAML → cards
|
||||
phase: 2
|
||||
priority: must-have
|
||||
depends_on: [A2, B1]
|
||||
- id: B3b
|
||||
title: Replan request on empty backlog; escalate on no-decompose
|
||||
phase: 2
|
||||
priority: should-have
|
||||
depends_on: [B3a]
|
||||
- id: G1
|
||||
title: PAUSE kill-switch + merge-gate honored before dispatch and merge
|
||||
phase: 2
|
||||
priority: must-have
|
||||
depends_on: [B2]
|
||||
- id: H1
|
||||
title: Cross-domain baseline persona library (exec, marketing, ops, research, assistant + engineering roles)
|
||||
phase: 1
|
||||
priority: must-have
|
||||
depends_on: [A1]
|
||||
- id: H2
|
||||
title: System-type profiles — declarative mapping of system type to persona roster + topology
|
||||
phase: 2
|
||||
priority: must-have
|
||||
depends_on: [H1]
|
||||
- id: H3
|
||||
title: System-type provisioning — user declares type; orchestrator instantiates the matching roster + structure
|
||||
phase: 2
|
||||
priority: must-have
|
||||
depends_on: [H2]
|
||||
- id: H4
|
||||
title: Update-surviving persona customization — ad-hoc edits/additions persisted in a PRESERVE-protected override layer (baseline merged with overrides)
|
||||
phase: 2
|
||||
priority: must-have
|
||||
depends_on: [H1]
|
||||
|
||||
assumptions:
|
||||
- id: ASM-1
|
||||
vetoable: true
|
||||
text: >-
|
||||
The Mosaic Backlog on the native Postgres storage service is the backlog
|
||||
of record.
|
||||
- id: ASM-2
|
||||
vetoable: true
|
||||
text: >-
|
||||
Claude gate roles have no native busy status, so readiness = pane-idle +
|
||||
heartbeat.
|
||||
- id: ASM-3
|
||||
vetoable: true
|
||||
text: 'Two-agent floor = 1 orchestrator + >=1 enhancer.'
|
||||
- id: ASM-4
|
||||
vetoable: true
|
||||
text: >-
|
||||
Baseline personas ship in framework/fleet/roles/ (reseeded on update);
|
||||
user overrides live in a separate PRESERVE_PATHS-protected layer and win
|
||||
on merge.
|
||||
|
||||
spend:
|
||||
advisory: true
|
||||
note: >-
|
||||
No per-task token meter yet; budgets degrade to TTL. Spend is tracked only
|
||||
as an advisory projection alongside each card.
|
||||
@@ -0,0 +1,109 @@
|
||||
# PRD — Mosaic Fleet Suite (init, configure, operate)
|
||||
|
||||
> **Workstream:** W-FLEET (Fleet) under mission `mvp-20260312` · **Phase:** 3→4 productization
|
||||
> **North star:** [docs/fleet/north-star.md](./north-star.md) · prior: Phase-2 observability (#579), durable launch (#581), real-agent enablement (#583/#584/#586), releases 0.0.35–0.0.37
|
||||
> **Lead:** Jarvis @ `w-jarvis`. **Collaborator:** coder agent @ `dragon-lin` (jwoltje@10.1.10.37:coder0-0).
|
||||
> Owner of this file: Fleet workstream lead. Does not modify MVP single-writer control-plane files.
|
||||
|
||||
## Mission
|
||||
|
||||
Turn the proven fleet primitives into a **user-installable, AI-free-configurable fleet product**:
|
||||
a user runs mosaic fleet init, answers a few questions (general / coding / research / hybrid),
|
||||
gets a recommended set of agents plus one always-on orchestrator wired for chat-ops, and can
|
||||
operate, mutate, re-create, and observe the fleet — over tmux today and Matrix tomorrow — from
|
||||
CLI/TUI and (designed-for) the webUI.
|
||||
|
||||
**Immediate tangible goal:** the **"Mos"** orchestrator agent running on `w-jarvis`, reachable
|
||||
in **Discord channel `1517622518662434996`** (server `1112631390438166618`). Once the fleet is
|
||||
functional, we use the fleet itself to continue the work.
|
||||
|
||||
## Requirements
|
||||
|
||||
### A. Configure-without-AI CLI
|
||||
|
||||
| ID | Requirement |
|
||||
| --- | ----------------------------------------------------------------------------------------------------------- |
|
||||
| R1 | mosaic fleet command set is functional end-to-end (init/install/start/stop/status/ps/verify + agent verbs). |
|
||||
| R2 | mosaic fleet init is an interactive, **AI-free** CLI wizard. |
|
||||
| R3 | Init asks the **configuration type**: `general`, `coding`, `research`, `hybrid`, … (extensible). |
|
||||
| R4 | Based on the answer, the fleet is populated with a **recommended set of agents** (a preset). |
|
||||
| R5 | **Exactly one main orchestrator agent** is always configured, regardless of type. |
|
||||
| R10 | A set of **recommended configurations (presets)** ships for easy duplication. |
|
||||
| R8 | User can **re-create** the fleet when config needs change (idempotent re-init / reconfigure). |
|
||||
| R17 | Fleet controls are **simple and intuitive**. |
|
||||
|
||||
### B. Comms & orchestrator chat-ops
|
||||
|
||||
| ID | Requirement |
|
||||
| --- | ----------------------------------------------------------------------------------------------------------------------------- |
|
||||
| R6 | Init can wire the orchestrator to a chat connector — **Telegram / Discord / Matrix / Slack** — for command + comms. |
|
||||
| R7 | Designed with the end-goal of **Matrix comms on a locally-controlled server**. |
|
||||
| R16 | Fleet supports **tmux AND Matrix** comms, **user-configurable** at init or any time. Not all users want Matrix. |
|
||||
| R19 | **"Mos" orchestrator on Discord** (chan 1517622518662434996 / srv 1112631390438166618) on `w-jarvis` — the first live target. |
|
||||
|
||||
### C. Runtime, health, lifecycle
|
||||
|
||||
| ID | Requirement |
|
||||
| --- | ---------------------------------------------------------------------------------- |
|
||||
| R9 | Fleet is **mutable by the orchestrator agent** — add/remove agents per need. |
|
||||
| R13 | Fleet **gracefully handles Pi + Claude harness updates** — keep harnesses current. |
|
||||
| R14 | The **Pi harness is customized** for proper tool usage, etc. |
|
||||
| R15 | **Agent heartbeat** properly configured for **Claude AND GPT/Pi** agents. |
|
||||
|
||||
### D. Surfaces, testing, docs
|
||||
|
||||
| ID | Requirement |
|
||||
| --- | ----------------------------------------------------------------------------------- |
|
||||
| R18 | Fleet built so the **webUI can view / monitor / terminate / butt-in** on a session. |
|
||||
| R11 | Installed and **tested on both `w-jarvis` and `dragon-lin`**. |
|
||||
| R12 | **Documentation**: how to install, configure, and use the fleet. |
|
||||
|
||||
## Architecture / approach
|
||||
|
||||
- **Config model:** `roster.yaml` is the source of truth (already exists). Add **presets** (`general`/`coding`/`research`/`hybrid`) as shipped example rosters; `init` selects a preset, always injects the orchestrator, and writes the roster. Re-init = regenerate roster (preserve user/site overrides — mirrors install env-merge from #567).
|
||||
- **Orchestrator agent:** always present; carries the chat connector config (connector type + target IDs) so it can be commanded over chat. tmux is the substrate; the connector bridges chat ↔ the orchestrator session.
|
||||
- **Comms layers (R16):** (1) **tmux** inter-agent (`agent-send`, proven) — default, always available. (2) **chat connector** for human↔orchestrator (Discord now; Matrix the strategic target). (3) **Matrix** as the locally-controlled cross-agent bus (future). Connector is pluggable + reconfigurable.
|
||||
- **Heartbeat (R15):** runtime-agnostic launcher sidecar already covers pi/claude/codex (#584). Refine per-runtime (native HB) with the **custom Pi harness** (R14) + a Claude path.
|
||||
- **Updates (R13):** mosaic update (CLI) + a fleet-aware harness-update step that refreshes pi/claude/codex and re-launches agents safely (drain → update → relaunch via the durable launcher).
|
||||
- **webUI (R18):** the fleet exposes machine-readable state (fleet ps --json already carries tenant/host/heartbeat/managed) + control verbs (start/stop/watch/send); webUI consumes these (control plane rides federation per north star). Ensure a stable JSON contract + a terminate/attach(butt-in) path.
|
||||
|
||||
## Phases (incremental, each shippable)
|
||||
|
||||
| Phase | Deliverable | Notes |
|
||||
| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
|
||||
| **F1 Presets + init wizard** | preset rosters (general/coding/research/hybrid) + always-orchestrator + AI-free fleet init selecting a preset; re-init idempotent | R1–R5, R8, R10, R17 |
|
||||
| **F2 Connector + Mos-on-Discord** | orchestrator chat-connector config (Discord first) + **Mos live on Discord 1517…/1112…** on w-jarvis | R6, R19, partial R16 |
|
||||
| **F3 Heartbeat + harness** | HB confirmed for claude + pi/gpt; **custom Pi harness** (tool usage, native HB, model self-report); graceful harness updates | R13, R14, R15 |
|
||||
| **F4 Matrix + comms toggle** | Matrix connector (local server) + user toggle tmux/Matrix at init/anytime | R7, R16 |
|
||||
| **F5 Orchestrator-mutable fleet** | orchestrator can add/remove agents at runtime | R9 |
|
||||
| **F6 webUI hooks** | stable JSON contract + terminate/attach surface for webUI view/monitor/terminate/butt-in | R18 |
|
||||
| **F7 Test + docs** | install+test on w-jarvis AND dragon-lin; user docs (install/configure/use) | R11, R12 (runs alongside every phase) |
|
||||
|
||||
## Work division (proposed — confirm with dragon-lin)
|
||||
|
||||
- **Jarvis @ w-jarvis (Lead):** F1 presets+wizard, F2 connector+Mos-on-Discord, F5 mutability, F6 webUI hooks; merge authority + dual-engine reviews; co-testing on w-jarvis.
|
||||
- **coder @ dragon-lin:** F3 custom Pi harness + harness-update flow (pi/codex-savvy); plus its in-flight constitution P4–P6 (P4 installer rework underpins fleet init/updates — coordinate the install path). Co-testing on dragon-lin (R11).
|
||||
- **Shared:** F4 Matrix (whoever has bandwidth); F7 testing/docs continuous.
|
||||
|
||||
## Immediate target: Mos on Discord (F2 first slice)
|
||||
|
||||
The discord plugin is available (~/.claude.json). Path: configure the **orchestrator** as a durable
|
||||
fleet session running Claude Code with the discord plugin bridged to channel `1517622518662434996`
|
||||
(server `1112631390438166618`) on w-jarvis, with the existing Discord Bridge Protocol (ack within
|
||||
~3s, reply via `mcp__discord__reply`, no `AskUserQuestion`). Heartbeat via the launcher sidecar.
|
||||
|
||||
## Success criteria
|
||||
|
||||
- A non-AI user can mosaic fleet init, pick a type, and get a working fleet + orchestrator.
|
||||
- **Mos answers in Discord 1517…** on w-jarvis.
|
||||
- Fleet runs + is observable (fleet ps) on **both** w-jarvis and dragon-lin.
|
||||
- Harness updates handled gracefully; HB healthy for claude + pi/gpt agents.
|
||||
- Docs let a new operator install/configure/use the fleet.
|
||||
- Re-init + orchestrator mutation work.
|
||||
|
||||
## Assumptions (veto-able)
|
||||
|
||||
- `ASSUMPTION:` presets ship as example rosters under the framework (fleet/examples/\*.yaml), selected by `init`.
|
||||
- `ASSUMPTION:` chat connectors are pluggable; Discord first (target exists), Matrix is the strategic default later.
|
||||
- `ASSUMPTION:` "Mos" = a Claude Code orchestrator session with the discord plugin (reuses the documented Discord Bridge Protocol).
|
||||
- `ASSUMPTION:` per north star, runtimes default to Codex/pi-on-Codex for workers; the orchestrator "Mos" runs Claude Code (in Claude Code, which is allowed).
|
||||
@@ -0,0 +1,109 @@
|
||||
# PRD — Fleet Phase 2: Operator Observability
|
||||
|
||||
> **Workstream:** W-FLEET under `mvp-20260312` · **Phase:** 2
|
||||
> **North star:** [docs/fleet/north-star.md](./north-star.md)
|
||||
> **Source umbrella PRD:** [docs/PRD.md](../PRD.md) (Mosaic Stack v0.1.0)
|
||||
> **Tracks task:** `fleet-observability-1` — restore operator observability into fleet agent sessions.
|
||||
|
||||
## Problem
|
||||
|
||||
The durable tmux fleet runs on the isolated `mosaic-fleet` socket. That isolation
|
||||
(which protects the operator's default tmux) makes the fleet **invisible** to default
|
||||
tooling, and truth is split across three planes no single command joins — systemd
|
||||
(systemctl --user), tmux (-L mosaic-fleet), and the process tree (`pstree`).
|
||||
agent tail (`capture-pane`) returns **blank for full-screen TUIs**, and agent send
|
||||
confirms only keystroke injection, not acceptance. Net: the operator has near-zero
|
||||
observability and no safe way to watch a session.
|
||||
|
||||
## Goals
|
||||
|
||||
1. One command shows the **whole fleet's** real state, joining all three planes.
|
||||
2. **Liveness is truthful**: healthy = answered a heartbeat, not "pane alive".
|
||||
3. The operator can **watch** any session read-only without disrupting it.
|
||||
4. `send` reports **delivered-and-accepted**, not just injected.
|
||||
5. Every record/address carries **`tenant_id` + `host`** (zero foreclosure for multi-tenant/multi-host).
|
||||
|
||||
## Non-goals (this phase)
|
||||
|
||||
- No webUI (Phase 5; rides federation for cross-host).
|
||||
- No `fleetd` daemon or persistent history store.
|
||||
- No real-runtime swap (Phase 3) — instrument the live **dogfood stub** fleet.
|
||||
- No cross-host aggregation yet (addressing is host-tagged but queries stay local).
|
||||
|
||||
## Functional requirements
|
||||
|
||||
| ID | Requirement |
|
||||
| ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| FR-1 | mosaic fleet ps [--json] prints one row per roster agent joining: name · tenant · host · runtime · systemd(active/enabled) · pane(alive/dead) · pid · idle · **last-heartbeat age** · **drift** flag (roster runtime ≠ actual pane command) · **boot-enable** warning (active but `UnitFileState=disabled`). |
|
||||
| FR-2 | **Heartbeat protocol v1** (see below); `dogfood-agent.py` implements the responder. fleet ps issues probes (or reads last-seen) and reports health per FR-1. |
|
||||
| FR-3 | mosaic agent watch <name> opens a **read-only** view of the pane (grouped session or tmux attach -r) that cannot send keystrokes and does not shrink the agent's window. |
|
||||
| FR-4 | mosaic agent attach <name> remains the **explicit** interactive-takeover path (separate verb, documented as the only one that can type). |
|
||||
| FR-5 | mosaic agent send <name> --verify confirms the message was **accepted** (not left as an unsubmitted draft) and returns non-zero if delivery cannot be verified. |
|
||||
| FR-6 | All structured output (`--json`) includes `tenant_id` and `host` fields. |
|
||||
|
||||
## Heartbeat protocol v1
|
||||
|
||||
- **Probe:** operator/fleet ps writes a sentinel line to the agent's input or a
|
||||
well-known per-agent heartbeat file path ~/.config/mosaic/fleet/run/<agent>.hb.
|
||||
- **Response:** the runtime updates <agent>.hb with ts=<iso8601> pid=<pid> status=<ok|busy>
|
||||
on a fixed interval (default 15s) and on demand when probed.
|
||||
- **Health rule:** `healthy` if now - ts <= 3 × interval; else `stale`; missing file = `unknown`.
|
||||
- **Contract:** every runtime (dogfood stub now; claude/codex/pi/opencode in Phase 3)
|
||||
MUST emit the heartbeat. The protocol is file-based so it works for headless stubs and
|
||||
full-screen TUIs alike (no `capture-pane` dependency).
|
||||
- `ASSUMPTION:` file-based heartbeat (vs in-pane echo) — chosen because it is TUI-safe and
|
||||
uid-scoped, fitting per-tenant isolation. Open to an OTEL-span variant in Phase 3 (MVP-X6).
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- mosaic fleet ps shows all 5 live sessions on `mosaic-fleet` with correct
|
||||
pane/pid/idle and flags the dogfood **drift** (`canary-pi` runtime=pi but pane runs
|
||||
`dogfood-agent.py`) and the **boot-enable** gap (active but disabled).
|
||||
- Killing one agent's pane flips its row to dead/stale within one `interval`.
|
||||
- agent watch shows live output and provably cannot type into the pane; detaching
|
||||
leaves the agent's window size unchanged.
|
||||
- agent send --verify returns success on an accepting pane and non-zero on a wedged/draft pane.
|
||||
- Quality gates green: pnpm typecheck, pnpm lint, pnpm format:check, plus
|
||||
pnpm --filter @mosaicstack/mosaic test.
|
||||
- Independent review passed; dogfood evidence captured against the live fleet.
|
||||
|
||||
## Test plan
|
||||
|
||||
- Unit/CLI specs in `packages/mosaic/src/commands/fleet.spec.ts` (and a new
|
||||
`fleet-ps`/`watch`/`send-verify` spec) using the injected `CommandRunner` to assert
|
||||
exact tmux/systemd command construction and JSON shape (tenant+host present).
|
||||
- Situational: run against the live `mosaic-fleet` fleet; capture fleet ps output,
|
||||
a kill-and-detect cycle, a read-only `watch`, and a send --verify pass/fail pair.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- **Verify heuristic is best-effort:** agent send --verify uses a > -prefix draft
|
||||
heuristic that is specific to pi/claude TUIs. Draft detection for codex and opencode
|
||||
TUIs is best-effort only; those runtimes may not use the same input-line indicator.
|
||||
- **Pane-change check is the best Phase-2 signal; verify now polls up to a bounded
|
||||
timeout:** agent send --verify captures a BEFORE snapshot, sends the message, then
|
||||
polls `capture-pane` every ~400 ms up to a configurable total timeout (default ~6 s,
|
||||
controlled by --verify-timeout <ms>). On each poll it runs classifySendResult: if
|
||||
the pane shows 'accepted' or 'draft' the loop exits immediately; while the result is
|
||||
'unverifiable' (no pane change yet) it keeps polling. After the timeout with no
|
||||
definitive result, it fails closed: exit 1 with "no pane change after send". This
|
||||
eliminates false 'unverifiable' failures for slow/loaded TUIs that were previously
|
||||
caused by the old fixed 300 ms single-capture. Definitive acceptance ultimately
|
||||
requires a runtime acknowledgement (Phase-3 heartbeat-ack); the bounded pane-change
|
||||
poll is the best signal available against an opaque TUI for Phase-2.
|
||||
- **Blank AFTER capture fails closed:** Full-screen TUIs (claude, codex, opencode, pi)
|
||||
render blank for tmux capture-pane. When the AFTER snapshot is empty, send --verify
|
||||
returns non-zero with an "unverifiable" message rather than silently succeeding. This
|
||||
is an intentional fail-closed design (FR-5).
|
||||
- **agent watch uses a grouped viewer session:** tmux attach -r directly against the
|
||||
agent session lets the viewer terminal shrink the agent's window. agent watch instead
|
||||
creates a throwaway grouped session (tmux new-session -d -t '=<agent>' -s
|
||||
'<agent>-watch-<pid>'), attaches read-only to that session, and kills it on detach.
|
||||
The grouped session shares the agent's windows but has independent sizing, so the
|
||||
agent's window is never affected. tmux attach is still interactive and requires
|
||||
inherited stdio; the `interactiveRunner` handles TTY passthrough.
|
||||
|
||||
## Surfaces & parity (MVP-X1)
|
||||
|
||||
CLI lands this phase. TUI surface follows in the `packages/mosaic` wizard; webUI in
|
||||
Phase 5 via federation. PRD records the parity debt explicitly so it is not lost.
|
||||
@@ -0,0 +1,63 @@
|
||||
# Fleet Configuration Management
|
||||
|
||||
This book documents the local roster-v2 desired-state control plane delivered under issue #758. The normative requirements are the [FCM section of the repository PRD](../PRD.md#fleet-declarative-configuration-management-workstream-fcm-758), not the older fleet-suite or observability planning pages.
|
||||
|
||||
## Authority boundary
|
||||
|
||||
<MOSAIC_HOME>/fleet/roster.yaml is the sole writable desired-state authority for local fleet membership, launch policy, and persisted lifecycle. Generated environment files, systemd enablement, tmux sessions, heartbeat files, and status output are derived or observed. Rebuild projections from the roster; never edit them as desired state.
|
||||
|
||||
This control plane is local tmux/systemd only. Remote/SSH entries and connectors are inventory, not reconciliation targets. Arbitrary commands, channels, secret references, gateway catalog convergence, and UI configuration storage are outside this workstream. `mos-comms` is temporary transport glue, not permanent fleet architecture.
|
||||
|
||||
## Choose the right workflow
|
||||
|
||||
1. **Need to inspect intent?** Read the roster and use mosaic fleet get; see [desired versus observed state](concepts/desired-vs-observed-state.md).
|
||||
2. **Need to inspect reality?** Use `status` or `doctor`; use `verify` for a strict non-zero drift/ownership gate. These commands do not repair anything.
|
||||
3. **Need to change membership or persisted policy?** Use generation-guarded `plan`, `create`, `update`, or `delete`; see [safe CRUD](how-to/create-update-delete-agent.md).
|
||||
4. **Need a one-time runtime action?** Use `start`, `stop`, or `restart`. These do not change persisted desired state.
|
||||
5. **Need convergence?** Review apply --dry-run, resolve blockers, then use `apply` with the same current generation; see [reconcile and recover](operations/reconcile-and-recover.md).
|
||||
6. **Need v1 migration evidence?** Use preview only. Cutover, canary, and rollback remain held for FCM-M4-002.
|
||||
7. **Need the gateway-backed agent catalog?** That is the separate mosaic agent surface, not local fleet desired state.
|
||||
|
||||
## Concepts
|
||||
|
||||
- [Desired versus observed state](concepts/desired-vs-observed-state.md)
|
||||
- [Identity, class, runtime, provider, and model](concepts/identity-class-runtime.md)
|
||||
- [Role authority and leases](concepts/role-authority-and-leases.md)
|
||||
- [Generated environment launch chain](concepts/generated-env-launch-chain.md)
|
||||
|
||||
## Operator how-to
|
||||
|
||||
- [Create, inspect, update, and delete](how-to/create-update-delete-agent.md)
|
||||
- [Start, stop, restart, and reconcile](how-to/start-stop-restart.md)
|
||||
- [Configure an interaction instance](how-to/configure-tess-interaction.md)
|
||||
- [Configure a validator instance](how-to/configure-ultron-validator.md)
|
||||
- [Customize roles](how-to/customize-roles.md)
|
||||
|
||||
## Operations and recovery
|
||||
|
||||
- [Reconcile and recover](operations/reconcile-and-recover.md)
|
||||
- [Environment quarantine](operations/env-quarantine.md)
|
||||
- [Systemd/tmux troubleshooting](operations/systemd-tmux-troubleshooting.md)
|
||||
- [Backup and restore boundary](operations/backup-restore.md)
|
||||
- [Upgrade and asset-drift hold](operations/upgrade-assets.md)
|
||||
|
||||
## Reference and migration
|
||||
|
||||
- [Roster v2 fields](reference/roster-v2-fields.md) · [executable JSON Schema](reference/roster-v2.schema.json) · [validated example](examples/roster-v2.yaml)
|
||||
- [CLI and exit codes](reference/cli.md)
|
||||
- [Role classes](reference/role-classes.md)
|
||||
- [Lifecycle transitions](reference/lifecycle-transitions.md)
|
||||
- [Status and drift](reference/status-and-drift.md)
|
||||
- [Generated environment boundary](reference/generated-env-boundary.md)
|
||||
- [v1-to-v2 preview](migration/v1-to-v2.md)
|
||||
- [Example/profile dispositions](migration/example-profile-disposition.md)
|
||||
- [Legacy class aliases](migration/legacy-class-aliases.md)
|
||||
|
||||
## Acceptance evidence and holds
|
||||
|
||||
- [M0/M5 IA checklist](FLEET-CONFIG-DOCS-IA-CHECKLIST.md)
|
||||
- [Legacy example/profile inventory](LEGACY-EXAMPLE-PROFILE-DISPOSITION-INVENTORY.md)
|
||||
- [M5 closure evidence](../reports/documentation/758-fleet-config-ia-closure.md)
|
||||
- [Approved-existing deferrals and live-action holds](../reports/deferred/758-fleet-config-deferrals.md)
|
||||
|
||||
The canonical publishing source remains this repository. This card does not publish externally, run a migration, operate a live fleet, or close parent issue #758.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Tasks — W-FLEET (Fleet) Phase 2: Observability
|
||||
|
||||
> Workstream task file for the Fleet. Single-writer: Fleet workstream lead (orchestrator).
|
||||
> Workers read but never modify. This is **not** the MVP rollup (`docs/TASKS.md`) — a
|
||||
> rollup row is proposed to the MVP orchestrator, not written here.
|
||||
>
|
||||
> Mission: `mvp-20260312` · PRD: [docs/fleet/PRD.md](./PRD.md) · North star: [docs/fleet/north-star.md](./north-star.md)
|
||||
> Status: `not-started` | `in-progress` | `done` | `blocked` | `failed`
|
||||
|
||||
| id | status | description | depends_on | agent | pr | notes |
|
||||
| ------------- | ----------- | ---------------------------------------------------------------------------------------------------------------- | --------------------- | ----------- | --- | --------------------------------------------------------------------------------------------------------------------------- |
|
||||
| FLEET-OBS-000 | done | Plan: north-star + Phase-2 PRD + workstream scaffolding | — | lead | — | persisted 2026-06-20 on `feat/fleet-observability` |
|
||||
| FLEET-OBS-001 | done | Heartbeat protocol v1 spec finalized in PRD + framework doc | FLEET-OBS-000 | lead | — | file-based ~/.config/mosaic/fleet/run/<agent>.hb; spec in PRD |
|
||||
| FLEET-OBS-002 | in-progress | Implement heartbeat responder in `dogfood-agent.py` | FLEET-OBS-001 | fleet-coder | — | dispatched to ad-hoc mosaic yolo fleet agent (dogfood) |
|
||||
| FLEET-OBS-003 | done | mosaic fleet ps — join systemd+tmux+proc+idle+heartbeat; tenant+host tagged; drift + boot-enable flags; `--json` | FLEET-OBS-001 | worker | — | commit ab47831; LIVE-verified on mosaic-fleet; caught canary-pi DRIFT + BOOT-ENABLE. Polish: idleSeconds parse returns null |
|
||||
| FLEET-OBS-004 | done | mosaic agent watch <name> — read-only join (no resize, no keystrokes) | FLEET-OBS-000 | worker | — | attach -r; verb wired |
|
||||
| FLEET-OBS-005 | done | mosaic agent send --verify — delivery/acceptance receipt | FLEET-OBS-000 | worker | — | --verify flag; draft-heuristic verify |
|
||||
| FLEET-OBS-006 | done | CLI specs for ps/watch/send-verify (tenant+host shape, command construction) | FLEET-OBS-003,004,005 | worker | — | 62 tests green (31 new); re-verified by lead |
|
||||
| FLEET-OBS-007 | not-started | Framework doc: fleet observability guide + verbs | FLEET-OBS-003,004,005 | lead | — | `docs/guides/` or `framework/tools/.../README` |
|
||||
| FLEET-OBS-008 | not-started | Independent review + dogfood verification on live fleet | FLEET-OBS-002..007 | reviewer | — | author ≠ reviewer; capture evidence in scratchpad |
|
||||
| FLEET-OBS-009 | not-started | Open PR → green CI (queue guard) → squash-merge → close `fleet-observability-1` | FLEET-OBS-008 | lead | — | trunk merge; no direct push to main |
|
||||
|
||||
## Proposed MVP rollup row (for the MVP orchestrator — not written by this workstream)
|
||||
|
||||
```text-table
|
||||
| W-FLEET | in-progress | Fleet (agent-session execution layer) | Phase 2/5 | docs/fleet/TASKS.md | observability dogfooded on live stub fleet; control plane rides federation (W1) |
|
||||
```
|
||||
@@ -0,0 +1,135 @@
|
||||
# Fleet Backlog Conventions
|
||||
|
||||
The **backlog** is Mosaic's native backlog-of-record for fleet work. It is built
|
||||
end-to-end on Mosaic's own storage layer (`@mosaicstack/db`, drizzle/Postgres)
|
||||
and surfaced as mosaic fleet backlog <sub> --json.
|
||||
|
||||
> **Mosaic-native, no Hermes.** This backlog REPLACES the former Hermes adapter.
|
||||
> There is **no** runtime dependency on Hermes, hermes kanban, or ~/.hermes
|
||||
> anywhere in this feature. Anything previously delegated to Hermes is recreated
|
||||
> here on Mosaic's own Postgres storage layer.
|
||||
|
||||
## Storage tier — PGlite by default, Postgres by config
|
||||
|
||||
The backlog uses the existing Mosaic storage layer; there is **no** new database
|
||||
engine (no sqlite, no raw client).
|
||||
|
||||
| Condition | Tier | Data location |
|
||||
| ---------------------------------- | -------------------- | ---------------------------------------------------------------- |
|
||||
| `DATABASE_URL` injected at runtime | Full server Postgres | the verified runtime database; it never authorizes migration/DDL |
|
||||
| `PGLITE_DATA_DIR` set (no URL) | Embedded PGlite | that directory |
|
||||
| neither (default) | Embedded PGlite | ~/.config/mosaic/fleet/backlog |
|
||||
|
||||
PGlite is real Postgres semantics in-process — including the row locks the atomic
|
||||
claim relies on — so the **same code** runs on a laptop (embedded, single-host
|
||||
default) and on a full Postgres deployment. Switching tiers is config-only.
|
||||
|
||||
For embedded PGlite only, the local backlog routine may prepare its local schema on first use. **Current operator behavior is PGlite-only.** The PostgreSQL path is held until KBN-101 activation; no current PostgreSQL CLI route, runner, or first-use migration is available or authorized. A future activated PostgreSQL runtime may connect only after its separately certified readiness gate.
|
||||
|
||||
### Update safety
|
||||
|
||||
The embedded PGlite store lives under ~/.config/mosaic/fleet/backlog, which is
|
||||
listed in `PRESERVE_PATHS` in `packages/mosaic/framework/install.sh`. This means
|
||||
mosaic update (which runs the framework sync with rsync --delete) will **not**
|
||||
wipe the operator's backlog — same protection as the roster, per-agent env, and
|
||||
heartbeat run dir.
|
||||
|
||||
## Card schema
|
||||
|
||||
A card is one row in the `backlog` table:
|
||||
|
||||
| Column | Type | Notes |
|
||||
| ------------------- | ------------------- | ------------------------------------------------------------- |
|
||||
| `id` | text (PK) | Stable, caller-supplied id (e.g. `A4`, `fleet-001`). |
|
||||
| `title` | text | Required. |
|
||||
| `body` | text (nullable) | Free-form description. |
|
||||
| `phase` | text (nullable) | Board/phase grouping (see below). |
|
||||
| `priority` | int (default 0) | **Higher = sooner.** Claim picks the max-priority ready card. |
|
||||
| `status` | enum | `ready` \| `claimed` \| `blocked` \| `done`. |
|
||||
| `depends_on` | jsonb string[] | DAG edges — ids of cards this one depends on. |
|
||||
| `claim_owner` | text (nullable) | Owner token of the active claim. |
|
||||
| `claim_ttl_seconds` | int (nullable) | TTL of the active claim. |
|
||||
| `claimed_at` | timestamptz (null) | When the claim was taken. claimed_at + ttl = expiry. |
|
||||
| `attempts` | int (default 0) | Incremented each time the card is claimed. |
|
||||
| `idempotency_key` | text (unique, null) | Dedups `create`; NULLs are distinct in Postgres. |
|
||||
| `acceptance` | jsonb (nullable) | Acceptance criteria (array of strings or object). |
|
||||
| `created_at` | timestamptz | |
|
||||
| `updated_at` | timestamptz | |
|
||||
|
||||
`depends_on` is modeled as a `jsonb` array column rather than a separate edge
|
||||
table. Justification: it matches the repo's existing style (e.g. `tasks.tags`,
|
||||
`agents.skills`, `routing_rules.conditions` are all jsonb arrays), keeps a card
|
||||
self-contained, and the DAG is small (per-card dependency lists), so a join table
|
||||
would add ceremony without benefit.
|
||||
|
||||
### Board / phase convention
|
||||
|
||||
`phase` is a free-form grouping string used as the board column / milestone label
|
||||
(e.g. `M1`, `fleet`, `infra`). list --phase <phase> filters to one board lane.
|
||||
`priority` orders cards **within** the ready pool regardless of phase.
|
||||
|
||||
## Status lifecycle
|
||||
|
||||
```text-diagram
|
||||
create
|
||||
│
|
||||
▼
|
||||
┌──────► ready ───── claim ─────► claimed ───── complete ─────► done
|
||||
│ │ │
|
||||
│ block reclaim (TTL expiry or --id)
|
||||
│ ▼ │
|
||||
│ blocked └──────────────────────────┘ (back to ready)
|
||||
└──────────┘ (reclaim / re-create can return a card to ready)
|
||||
```
|
||||
|
||||
- **ready** — eligible to be claimed once every `depends_on` card is `done`.
|
||||
- **claimed** — a worker holds it; `claim_owner` + `claimed_at` set.
|
||||
- **blocked** — explicitly parked; never auto-claimed.
|
||||
- **done** — completed; satisfies dependents.
|
||||
|
||||
## Atomic claim (FOR UPDATE SKIP LOCKED) + TTL
|
||||
|
||||
`claim` is atomic. Inside a single transaction it locks candidate `ready` rows
|
||||
with SELECT ... FOR UPDATE SKIP LOCKED (via the drizzle `sql` operator), picks
|
||||
the highest-priority deps-satisfied card, and flips it to `claimed`. Because a row
|
||||
already locked by a concurrent claimer is **skipped**, two claimers can **never**
|
||||
both win the same card — the loser falls through to the next candidate or gets
|
||||
`null`. (Proven by the concurrency tests in `packages/db/src/backlog.spec.ts`.)
|
||||
|
||||
- **Deps gate:** a card is only claimable when every id in `depends_on` is `done`.
|
||||
- **TTL:** claim --ttl <sec> (default **900s**) records `claim_ttl_seconds`.
|
||||
- **reclaim:** releases claims whose claimed_at + ttl is in the past (expired)
|
||||
back to `ready`, clearing the claim fields. reclaim --id <id> force-releases a
|
||||
specific card regardless of expiry. This is how a crashed worker's card returns
|
||||
to the pool.
|
||||
|
||||
## CLI — mosaic fleet backlog <sub> --json
|
||||
|
||||
All subcommands support `--json`.
|
||||
|
||||
| Subcommand | Purpose |
|
||||
| ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| create --id --title [--body --phase --priority --depends-on --acceptance --idempotency-key] | Create a card; `idempotency_key` dedups (repeat returns the existing card). |
|
||||
| list [--status --phase --ready-only] | List cards. `--ready-only` = status `ready` AND all deps `done`. |
|
||||
| claim --owner [--ttl <sec> --id <id>] | Atomically claim the highest-priority ready card (or `--id`). Returns the card or `null`. |
|
||||
| reclaim [--id <id>] | Release expired claims (or a specific card) back to `ready`. |
|
||||
| link --from --to | Add a `depends_on` edge (`--from` depends on `--to`). |
|
||||
| `stats` | Counts by status, oldest-ready age, expired-claim count. |
|
||||
| block --id | Set a card to `blocked`. |
|
||||
| complete --id | Set a card to `done` (releases any claim). |
|
||||
|
||||
### Example
|
||||
|
||||
Seed two cards; the second depends on the first. Because A2 is gated on A1, claim returns A1 first. Finish A1, then list A2 as ready. Recover stalled work.
|
||||
|
||||
```fleet-command
|
||||
mosaic fleet backlog create --id A1 --title "schema" --priority 5
|
||||
mosaic fleet backlog create --id A2 --title "service" --depends-on A1 --priority 9
|
||||
|
||||
mosaic fleet backlog claim --owner worker-1 --ttl 600 --json
|
||||
|
||||
mosaic fleet backlog complete --id A1
|
||||
mosaic fleet backlog list --ready-only --json
|
||||
|
||||
mosaic fleet backlog reclaim --json
|
||||
```
|
||||
@@ -0,0 +1,42 @@
|
||||
# Desired, Derived, and Observed Fleet State
|
||||
|
||||
## One writable authority
|
||||
|
||||
The canonical local v2 roster at <MOSAIC_HOME>/fleet/roster.yaml is desired state. Membership, stable identity, class, runtime/provider/model selection, launch policy, enablement, and persisted `running`/`stopped` intent are written only through generation-guarded roster mutations.
|
||||
|
||||
Derived projections are reproducible consequences of that authority:
|
||||
|
||||
- <name>.env.generated;
|
||||
- exact roster-named tmux sessions on the configured socket after reconciliation;
|
||||
- systemd service targets managed by installation/reconciliation.
|
||||
|
||||
Current systemd unit enablement is not yet lifecycle-conformant at boot: installation can enable every
|
||||
agent unit, and the launcher projection does not carry `enabled` or `desired_state`. Therefore reboot
|
||||
preservation for stopped/disabled agents remains an FCM-M3-002 acceptance hold, not a guaranteed
|
||||
projection behavior.
|
||||
|
||||
Observed evidence available to current roster-v2 status commands includes systemd active state, tmux
|
||||
presence, holder ownership, and unmanaged sessions. Heartbeat files are observational in the wider fleet,
|
||||
but roster-v2 `status`, `doctor`, and `verify` do not currently read them. Observation never writes back
|
||||
to the roster.
|
||||
|
||||
## Generation and ownership
|
||||
|
||||
`generation` is a positive integer concurrency fence. A mutating request must provide the current value. Successful changed CRUD increments it exactly once; stale or concurrent writers fail before mutation. Apply/reconcile rereads the canonical roster under a private exclusive lock and uses only that generation and content for effects.
|
||||
|
||||
Ownership is exact, never fuzzy. Reconciliation is limited to roster names, the configured socket, the exact holder session, a private installation identity, and private managed paths. An ownership mismatch, unmanaged session, unsafe path, stale generation, or ambiguous lock fails closed.
|
||||
|
||||
## Drift decisions
|
||||
|
||||
| Observation | Interpretation | Safe response |
|
||||
| ---------------------------------------- | ------------------------ | ---------------------------------------------------------------------- |
|
||||
| Generated file differs or is missing | Derived projection drift | Review apply --dry-run; regenerate from the roster. |
|
||||
| Desired `running`, exact session missing | `missing-session` | Diagnose ownership/runtime, then reconcile if safe. |
|
||||
| Desired `stopped`, exact session present | `unexpected-session` | Inspect; reconciliation may stop only the proven roster target. |
|
||||
| Disabled agent running | `disabled-running` | Inspect; disabled state wins during explicit safe reconciliation. |
|
||||
| Unknown session on the configured socket | Unmanaged state | Report only. Do not adopt, rename, or kill it. |
|
||||
| Heartbeat stale in the wider fleet | Liveness evidence | Diagnose separately; current roster-v2 status does not read heartbeat. |
|
||||
|
||||
`status` and `doctor` classify. `verify` is also observational but exits non-zero when ownership, drift, or unmanaged-state checks fail. `plan`/apply --dry-run validates proposed projection and lifecycle work without mutation. `apply` and `reconcile` converge only after all preconditions pass.
|
||||
|
||||
A partial projection failure does not roll the roster back. Treat the committed roster as authority and regenerate. A lifecycle failure after projection completion preserves both roster and projections for inspection. Sensitive legacy values are never printed; diagnostics are bounded to stable codes, key names where applicable, and hashes.
|
||||
@@ -0,0 +1,23 @@
|
||||
# Generated Environment Launch Chain
|
||||
|
||||
The launcher consumes validated data, not shell configuration.
|
||||
|
||||
1. Read and validate the canonical roster.
|
||||
2. Render deterministic <name>.env.generated data from that roster.
|
||||
3. Parse optional <name>.env.local through a strict allowlist.
|
||||
4. Reject generated-key shadowing, unknown or sensitive-looking keys, unsafe paths/values, duplicates, malformed lines, shell syntax, and command overrides.
|
||||
5. Derive the runtime command from validated runtime/model/reasoning data.
|
||||
6. Target only the exact configured tmux socket and roster session after ownership checks.
|
||||
|
||||
## File precedence and ownership
|
||||
|
||||
| File | Owner | Use |
|
||||
| ----------------- | ------------------------ | --------------------------------------------------------------------------- |
|
||||
| `.env.generated` | Mosaic projection writer | Complete deterministic roster projection. Rebuild; do not edit. |
|
||||
| `.env.local` | Operator | Optional, private, strict machine-local data. Cannot shadow generated keys. |
|
||||
| `.env` | Legacy input | One-time migration input only; never launch authority. |
|
||||
| `.env.quarantine` | Private quarantine | Retained unsafe legacy evidence; never loaded by the launcher. |
|
||||
|
||||
Neither systemd nor the launcher sources these files. No `eval`, shell expansion, arbitrary `MOSAIC_AGENT_COMMAND`, channel, or secret-reference compatibility path exists. Safe legacy generated keys are regenerated, allowed local keys are relocated, and unsafe material is quarantined.
|
||||
|
||||
Diagnostics never expose the rejected value, credential material, or command text. They are bounded to stable rule code, key name where safe, and SHA-256 content identity. See [generated environment reference](../reference/generated-env-boundary.md) and [quarantine operations](../operations/env-quarantine.md).
|
||||
@@ -0,0 +1,20 @@
|
||||
# Fleet Identity, Class, and Runtime
|
||||
|
||||
Each roster field has one job. Do not use names or model strings as authority shortcuts.
|
||||
|
||||
| Concern | Field | Contract |
|
||||
| ----------------------- | ----------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| Stable machine identity | agents[].name | Unique, immutable mutation target and exact service/session name. |
|
||||
| Display identity | agents[].alias | Human-facing label only; may be changed and grants no authority. |
|
||||
| Behavioral contract | agents[].class | Resolves through the shared baseline plus `roles.local` persona library. |
|
||||
| Tool boundary | agents[].tool_policy | Must match protected canonical classes; cannot independently grant authority. |
|
||||
| Harness | agents[].runtime | One of `claude`, `codex`, `opencode`, or `pi`, declared in `runtimes`. |
|
||||
| Backend selection | agents[].provider and `model` | Explicit non-empty data; capability validity is not inferred from the display name or class. |
|
||||
| Effort | agents[].reasoning | `low`, `medium`, or `high`. |
|
||||
| Local placement | `working_directory` | Explicit safe local work path; not remote placement authority. |
|
||||
|
||||
Tess and Ultron are conventional instance/display names only. They are not products, required machine identities, role aliases, or authority-bearing classes. A configurable interaction instance uses class: interaction; a configurable validation instance uses class: validator. Any stable name and alias satisfying the structural contract may be used.
|
||||
|
||||
Class aliases are deliberately narrow: implementer → code, reviewer → review, and operator-interaction → interaction. No runtime, provider, model, persona prose, or instance name changes this mapping. See [role classes](../reference/role-classes.md) and the [validated generic example](../examples/roster-v2.yaml).
|
||||
|
||||
Roster v2 is local-only. It contains no host/SSH placement, connector, channel, secret-reference, arbitrary-command, per-agent socket, or gateway mapping fields. Those concerns require separate requirements and threat models.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Fleet Role Authority and Leases
|
||||
|
||||
Role content describes behavior; protected authority is immutable code metadata derived only from the canonical class.
|
||||
|
||||
## Required workstream classes
|
||||
|
||||
`code`, `review`, `validator`, `orchestrator`, `team-leader`, `enhancer`, and `interaction` are required FCM classes. `merge-gate` is additionally protected because it remains the sole approve-to-land and merge authority.
|
||||
|
||||
| Class | Authority | Boundary |
|
||||
| -------------------------------------------- | ------------------------------------------------- | -------------------------------------------------------------------------------------- |
|
||||
| `merge-gate` | Approve-to-land and merge | Sole merge authority. |
|
||||
| `validator` | Issue independent validation evidence/certificate | Never approves landing or merges. |
|
||||
| `orchestrator` | Orchestrate topology and issue bounded leases | Does not gain merge authority. |
|
||||
| `team-leader` | Use explicitly leased capacity | Cannot issue leases or mutate roster, credentials, topology authority, or merge state. |
|
||||
| `interaction` | Receive requests and report status | Cannot orchestrate, issue leases, mutate configuration, or merge. |
|
||||
| `code`, `review`, `enhancer`, custom classes | No protected authority by default | Persona prose cannot grant protected powers. |
|
||||
|
||||
A lease is capacity authorization from an orchestrator, not ownership. It must identify a bounded task or period and does not alter the leased agent's roster identity, role contract, credentials, authority, or persisted lifecycle. Expiry/revocation returns capacity; it does not rewrite the roster.
|
||||
|
||||
Semantic validation rejects protected class/tool-policy mismatch in either direction. An instance named Ultron with class: validator remains validation-only. An instance named Tess with class: interaction remains request/status-only. Renaming either instance changes no authority.
|
||||
|
||||
For resolver layering and safe customization, see [role classes](../reference/role-classes.md) and [customize roles](../how-to/customize-roles.md).
|
||||
@@ -0,0 +1,61 @@
|
||||
version: 2
|
||||
generation: 1
|
||||
transport: tmux
|
||||
tmux:
|
||||
socket_name: mosaic-fleet
|
||||
holder_session: _holder
|
||||
defaults:
|
||||
working_directory: ~/src
|
||||
runtime: pi
|
||||
runtimes:
|
||||
pi:
|
||||
reset_command: /new
|
||||
agents:
|
||||
- name: code-example
|
||||
alias: Code Example
|
||||
class: code
|
||||
runtime: pi
|
||||
provider: example-provider
|
||||
model: example-model
|
||||
reasoning: medium
|
||||
tool_policy: code
|
||||
working_directory: ~/src
|
||||
persistent_persona: false
|
||||
reset_between_tasks: true
|
||||
lifecycle:
|
||||
enabled: true
|
||||
desired_state: stopped
|
||||
launch:
|
||||
yolo: false
|
||||
- name: interaction-example
|
||||
alias: Interaction Example
|
||||
class: interaction
|
||||
runtime: pi
|
||||
provider: example-provider
|
||||
model: example-model
|
||||
reasoning: low
|
||||
tool_policy: interaction
|
||||
working_directory: ~/src
|
||||
persistent_persona: true
|
||||
reset_between_tasks: false
|
||||
lifecycle:
|
||||
enabled: true
|
||||
desired_state: stopped
|
||||
launch:
|
||||
yolo: false
|
||||
- name: validator-example
|
||||
alias: Validator Example
|
||||
class: validator
|
||||
runtime: pi
|
||||
provider: example-provider
|
||||
model: example-model
|
||||
reasoning: high
|
||||
tool_policy: validator
|
||||
working_directory: ~/src
|
||||
persistent_persona: false
|
||||
reset_between_tasks: true
|
||||
lifecycle:
|
||||
enabled: true
|
||||
desired_state: stopped
|
||||
launch:
|
||||
yolo: false
|
||||
@@ -0,0 +1,92 @@
|
||||
# F4 — Orchestrator chat connector + Matrix (local homeserver)
|
||||
|
||||
> **Issue:** #616 · **Doctrine:** `docs/fleet/north-star.md` (#613) — orchestrator-chat-connector decision.
|
||||
> **Status:** Phase 1 (abstraction + scaffold) in this PR; Phase 2+ are follow-ups (below).
|
||||
|
||||
## Goal
|
||||
|
||||
The fleet **orchestrator** is the operator's single point of contact. The north-star makes the
|
||||
chat channel a **user-chosen connector** — tmux today, Discord live ("Mos"), with Matrix /
|
||||
Telegram / Slack configurable. F4 adds **Matrix** (local homeserver) as a **peer** connector and,
|
||||
first, the small **connector abstraction** that makes connectors pluggable without touching fleet
|
||||
core.
|
||||
|
||||
## The abstraction (Phase 1 — this PR)
|
||||
|
||||
Connectors implement one small, uniform interface (`src/fleet/connectors/types.ts`):
|
||||
|
||||
```typescript
|
||||
interface OrchestratorConnector {
|
||||
readonly kind: 'tmux' | 'discord' | 'matrix';
|
||||
send(message: OutboundMessage): Promise<SendResult>; // orchestrator → human
|
||||
subscribe(handler: (m: InboundMessage) => void): Unsubscribe; // human → orchestrator
|
||||
health(): Promise<ConnectorHealth>; // reachable + authenticated
|
||||
}
|
||||
```
|
||||
|
||||
- **send / subscribe / health** — the only surface fleet core depends on. `SendResult` is the
|
||||
ack half; health() is the liveness half.
|
||||
- **Thread-aware by metadata** — `OutboundMessage.threadId` / `InboundMessage.threadId` are
|
||||
optional, so thread-capable connectors (Matrix rooms/threads, the future first-party Mosaic
|
||||
Discord plugin) fit **without an interface change**.
|
||||
- **Registry** (`registry.ts`) — implementations register a factory by kind; createConnector(config)
|
||||
resolves one from roster config. Phase 1 ships the registry + `resolveConnectorKind` (defaults
|
||||
`tmux` when a roster declares no connector — **back-compat**); the factories land in Phase 2.
|
||||
|
||||
### Config model
|
||||
|
||||
A roster may carry an optional `connector` block (`roster.schema.json`); absent ⇒ tmux.
|
||||
|
||||
```yaml
|
||||
connector:
|
||||
kind: matrix
|
||||
matrix:
|
||||
homeserver_url: https://matrix.example.internal
|
||||
user_id: '@mos:example.internal'
|
||||
room_id: '!abc:example.internal'
|
||||
```
|
||||
|
||||
**Secrets are never in the roster.** `MATRIX_ACCESS_TOKEN` / `DISCORD_BOT_TOKEN` come from the
|
||||
environment (the gateway env-config pattern that already masks them). The sanitization gate would
|
||||
reject a token committed to a shipped file anyway.
|
||||
|
||||
## Matrix connector (Phase 2)
|
||||
|
||||
The connector speaks the **Matrix client-server API** directly over HTTPS (`fetch` — no SDK needed
|
||||
for MVP), so it is **homeserver-agnostic**:
|
||||
|
||||
| Op | Matrix CS-API |
|
||||
| ----------- | ----------------------------------------------------------------------- |
|
||||
| `send` | PUT /\_matrix/client/v3/rooms/{roomId}/send/m.room.message/{txnId} |
|
||||
| `subscribe` | GET /\_matrix/client/v3/sync (long-poll, `since` token) → room timeline |
|
||||
| `health` | GET /\_matrix/client/versions (reachable) + …/account/whoami (authed) |
|
||||
| threads | `m.thread` relations ↔ `threadId` |
|
||||
|
||||
## Local homeserver (infra, not connector code)
|
||||
|
||||
Strategic default: a **self-hosted** homeserver on our own infra — no third-party gateway.
|
||||
|
||||
- **Default: Conduit** (Rust, single binary, low resource) — trivial to stand up for a fleet/dev
|
||||
homeserver.
|
||||
- **Alternative: Synapse** (mature, feature-complete) for scale.
|
||||
|
||||
The connector only needs `homeserver_url` + `user_id` + `room_id` + an access token, so the
|
||||
homeserver choice is a **deployment** concern (a Phase-2 deploy guide), not connector code.
|
||||
|
||||
## Phasing
|
||||
|
||||
| Phase | Scope | This PR |
|
||||
| ----- | --------------------------------------------------------------------------------------- | ------- |
|
||||
| **1** | Connector interface + types, registry + kind resolution, roster `connector` schema, doc | ✅ yes |
|
||||
| 2 | Matrix CS-API client (fetch-based send/sync/health) + registered factory + tests | follow |
|
||||
| 2 | fleet init / `configure` connector-selection UX; roster parse wires the block | follow |
|
||||
| 2 | systemd launch wiring so the orchestrator starts on the chosen connector | follow |
|
||||
| 3 | Conduit deploy guide; first-party Mosaic Discord (threads) registers as a connector | follow |
|
||||
|
||||
## Back-compat & boundaries
|
||||
|
||||
- Existing rosters (no `connector`) resolve to tmux — **zero change**.
|
||||
- Fleet core never branches on connector kind; it depends only on the interface.
|
||||
- Cross-host reach rides the **federation** layer (W1), not a bespoke broker (north-star assumption).
|
||||
- Phase 1 touches **no** `fleet.ts` core (a self-contained `connectors/` module), so it is
|
||||
independent of the in-flight fleet-config PRs.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Configure an Interaction Instance
|
||||
|
||||
An interaction instance is a configurable local roster member with canonical class: interaction and matching tool_policy: interaction. “Tess” may be used as a display alias, but neither that alias nor the stable name is required or authority-bearing.
|
||||
|
||||
Use the [validated generic roster](../examples/roster-v2.yaml) as the safe shape. Choose a unique stable `name`, any descriptive `alias`, a supported declared runtime, explicit provider/model/reasoning, and a safe work directory. Start with:
|
||||
|
||||
```yaml
|
||||
name: interaction-example
|
||||
alias: Interaction Example
|
||||
class: interaction
|
||||
tool_policy: interaction
|
||||
lifecycle:
|
||||
enabled: true
|
||||
desired_state: stopped
|
||||
```
|
||||
|
||||
Plan the complete agent payload with the current roster generation, then create it without `--persisted-start`. Creation defaults to enabled/stopped and performs no runtime action. Review the resulting roster and projection plan before any later lifecycle decision.
|
||||
|
||||
The interaction class is request/status only. It cannot orchestrate, issue leases, mutate the roster/configuration, grant credentials, certify validation, approve landing, or merge. Connector and channel configuration are outside roster v2; do not add connector, channel, secret, command, remote-host, or gateway fields.
|
||||
|
||||
See [safe CRUD](create-update-delete-agent.md), [identity separation](../concepts/identity-class-runtime.md), and [role authority](../concepts/role-authority-and-leases.md).
|
||||
@@ -0,0 +1,21 @@
|
||||
# Configure a Validator Instance
|
||||
|
||||
A validator instance is a configurable local roster member with canonical class: validator and matching tool_policy: validator. “Ultron” may be used as a display alias, but it is not a required identity, class alias, product name, or source of authority.
|
||||
|
||||
Use the [validated generic roster](../examples/roster-v2.yaml) as the safe shape. Choose a unique stable name and explicit supported runtime/provider/model/reasoning values. Start stopped:
|
||||
|
||||
```yaml
|
||||
name: validator-example
|
||||
alias: Validator Example
|
||||
class: validator
|
||||
tool_policy: validator
|
||||
lifecycle:
|
||||
enabled: true
|
||||
desired_state: stopped
|
||||
```
|
||||
|
||||
Plan the full payload with the current generation and create without `--persisted-start`. Creation writes desired state and projections only; it does not launch a validator.
|
||||
|
||||
`validator` may issue independent validation evidence or a certificate. It has no approve-to-land or merge authority. `merge-gate` remains the sole protected merge authority, and changing the validator's name, alias, persona prose, runtime, provider, model, or tool-policy text cannot elevate it.
|
||||
|
||||
Certificate consumption and final release evidence remain FCM-M5-002 gates. This page does not create a certificate or authorize merge. See [safe CRUD](create-update-delete-agent.md) and [role authority](../concepts/role-authority-and-leases.md).
|
||||
@@ -0,0 +1,74 @@
|
||||
# Create, Inspect, Update, and Delete a Local Fleet Agent
|
||||
|
||||
Use the local roster-v2 control plane only. These commands change desired state and derived environment projections; they never start, stop, reconcile, inspect, or otherwise act on systemd, tmux, sessions, or runtimes.
|
||||
|
||||
## Read and plan first
|
||||
|
||||
```fleet-synopsis
|
||||
mosaic fleet get <name>
|
||||
mosaic fleet plan create --expected-generation <n> --agent '<json>'
|
||||
mosaic fleet plan update <name> --expected-generation <n> --agent '<json>'
|
||||
mosaic fleet plan delete <name> --expected-generation <n>
|
||||
```
|
||||
|
||||
plan create takes the name from `--agent`. plan update and plan delete require the target name immediately after the operation. A plan is deterministic and side-effect free: it validates the complete proposed roster and projection targets without changing files. Use `--dry-run` on `create`, `update`, or `delete` for the same no-write result.
|
||||
|
||||
Every successful command prints JSON. `get` returns { "generation", "agent" }; mutation results contain `plan`, `applied`, `authoritativeRoster`, and `projections`.
|
||||
|
||||
## Create safely
|
||||
|
||||
```fleet-command
|
||||
mosaic fleet create --expected-generation 7 --agent '{
|
||||
"name":"coder0",
|
||||
"alias":"Coder 0",
|
||||
"className":"code",
|
||||
"runtime":"pi",
|
||||
"provider":"openai",
|
||||
"model":"gpt-5.6-sol",
|
||||
"reasoning":"high",
|
||||
"toolPolicy":"code",
|
||||
"workingDirectory":"/srv/mosaic",
|
||||
"persistentPersona":false,
|
||||
"resetBetweenTasks":true,
|
||||
"launch":{"yolo":true}
|
||||
}'
|
||||
```
|
||||
|
||||
Create defaults to enabled: true and desired_state: stopped. It does not start a process. Add `--persisted-start` only to persist desired_state: running; that still does not start a runtime in this M2 command. The JSON payload is an allowlist of the roster-v2 fields shown above plus `launch.yolo`; command, channel, secret-reference, and other unknown keys are rejected rather than ignored. The JSON error exposes only a stable code, never the rejected value.
|
||||
|
||||
## Update and delete safely
|
||||
|
||||
```fleet-synopsis
|
||||
mosaic fleet update <name> --expected-generation <n> --agent '<complete JSON agent payload>'
|
||||
mosaic fleet delete <name> --expected-generation <n>
|
||||
```
|
||||
|
||||
Updates require a complete agent JSON payload and preserve the stable name. Delete removes only the exact roster-owned `coder0.env.generated` projection. It retains `coder0.env.local`, legacy `coder0.env`, `coder0.env.quarantine`, and every unrelated projection. A delete dry-run leaves all of those files byte-identical.
|
||||
|
||||
## Handle generation conflicts
|
||||
|
||||
Every mutation requires the current authoritative `--expected-generation`. A stale value returns JSON error.code: "stale-generation" with a non-zero exit. Reload with mosaic fleet get <name> or reread the roster, plan again using the returned generation, then retry. A concurrent mutation returns `concurrent-mutation`; do not force or bypass the lock.
|
||||
|
||||
## Interpret partial failures
|
||||
|
||||
The roster is authoritative and is written before derived projections. A late projection I/O failure returns non-zero with redacted, actionable JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"applied": false,
|
||||
"authoritativeRoster": "committed",
|
||||
"projections": "incomplete",
|
||||
"recovery": {
|
||||
"code": "projection-apply-failed",
|
||||
"action": "regenerate-projections-from-roster"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This is not a rollback and not a no-op: reload the roster because its generation and membership were committed, regenerate projections from that roster, then plan a new mutation. Recovery output never contains environment values, credentials, or command text.
|
||||
|
||||
## Exit and boundary behavior
|
||||
|
||||
Handled validation errors and partial projection failures exit non-zero. `plan`/`--dry-run` and normal mutation JSON make the state explicit; scripts should use both the exit code and `authoritativeRoster`/`projections`, not `applied` alone.
|
||||
|
||||
The commands operate only on <mosaic-home>/fleet/roster.yaml, the local roster desired-state authority. They do not accept arbitrary commands, channels, secrets, remote/connector actions, migration/canary actions, or runtime lifecycle operations.
|
||||
@@ -0,0 +1,54 @@
|
||||
# Customize Fleet Roles
|
||||
|
||||
Mosaic resolves persona contracts through two layers:
|
||||
|
||||
1. fleet/roles/<canonical-class>.md — seeded baseline contract.
|
||||
2. fleet/roles.local/<canonical-class>.md — operator override or custom role; this layer wins.
|
||||
|
||||
The same shared resolver is used by profile validation, provisioning, roster-v2 semantic validation,
|
||||
and launch-time persona injection.
|
||||
|
||||
## Override a baseline role
|
||||
|
||||
Create a readable Markdown contract under `roles.local` with the canonical filename and class marker:
|
||||
|
||||
```markdown
|
||||
# Code — local role definition
|
||||
|
||||
The local code role (`class: code`) follows the operator's repository conventions.
|
||||
```
|
||||
|
||||
Save it as `fleet/roles.local/code.md`. Do not edit generated or seeded baseline assets when the goal
|
||||
is a durable local customization.
|
||||
|
||||
Legacy aliases canonicalize before lookup. Therefore `roles.local/implementer.md` does not override
|
||||
`code`; use `roles.local/code.md`. See [Legacy Fleet Class Aliases](../migration/legacy-class-aliases.md).
|
||||
|
||||
## Add a custom class
|
||||
|
||||
A custom class remains supported when a readable contract exists for the exact identifier:
|
||||
|
||||
```markdown
|
||||
# Release notes — local role definition
|
||||
|
||||
The release-notes role (`class: release-notes`) prepares operator-reviewed release copy.
|
||||
```
|
||||
|
||||
Save it as `fleet/roles.local/release-notes.md`, then reference class: release-notes and a matching
|
||||
tool_policy: release-notes in roster v2. Adding only a `LIBRARY.md` row is insufficient.
|
||||
|
||||
Names such as `worker`, `analyst`, and `canary` are not built-in aliases; they need genuine custom
|
||||
contracts. agents[].alias, Tess, and Ultron are display names and cannot select a class.
|
||||
|
||||
## Validation and authority boundaries
|
||||
|
||||
Semantic validation reads the winning contract and rejects missing, unreadable, or empty files.
|
||||
Protected authority is derived from canonical class metadata in code, never from role prose. A custom
|
||||
contract cannot claim merge, validation-certificate, orchestration, lease, or interaction authority.
|
||||
|
||||
Roster v2 also fails closed when a protected class and tool policy do not match after canonicalization,
|
||||
or when an unprotected class claims a protected tool policy. The legacy `operator-interaction` policy
|
||||
canonicalizes to `interaction`.
|
||||
|
||||
Role customization does not issue leases, store validation certificates, mutate credentials, or
|
||||
change lifecycle state.
|
||||
@@ -0,0 +1,23 @@
|
||||
# Safely Reconcile and Control a Local Fleet Agent
|
||||
|
||||
Use the canonical local roster-v2 command surface:
|
||||
|
||||
```fleet-synopsis
|
||||
mosaic fleet apply --expected-generation <n> --dry-run
|
||||
mosaic fleet apply --expected-generation <n>
|
||||
mosaic fleet reconcile --expected-generation <n>
|
||||
mosaic fleet start <name> --expected-generation <n>
|
||||
mosaic fleet stop <name> --expected-generation <n>
|
||||
mosaic fleet restart <name> --expected-generation <n>
|
||||
mosaic fleet status [<name>]
|
||||
mosaic fleet verify
|
||||
mosaic fleet doctor
|
||||
```
|
||||
|
||||
Start with `--dry-run`. It validates roster semantics, deterministic projections, private managed paths, exact holder ownership, and named-socket state without changing files or lifecycle state. Explicit `apply` and `reconcile` rebuild derived projections and enforce persisted roster state: enabled `running` agents may start, while stopped or disabled agents are not started. This guarantee does not extend to reboot/service activation yet; boot preservation remains an FCM-M3-002 hold.
|
||||
|
||||
`start`, `stop`, and `restart` are explicit one-shot exact-service actions. They do not persist a lifecycle change. `update` preserves the agent's existing lifecycle, and no delivered operation changes durable lifecycle after creation.
|
||||
|
||||
Every command prints JSON. Observation commands report drift without mutation; `verify` exits non-zero on ownership mismatch, unmanaged sessions, or drift. A failed apply that wrote some derived projections reports projections: "incomplete" with bounded recovery to regenerate from the roster. A lifecycle failure after projections reports incomplete lifecycle work; it is never represented as a rollback or no-op.
|
||||
|
||||
These commands are local only. Remote/SSH/connector entries are inventory/validation-only. Commands do not accept arbitrary runtime commands, channels, secrets, generated-file desired state, or arbitrary tmux sockets.
|
||||
@@ -0,0 +1,69 @@
|
||||
# Executable Fleet Example, Profile, and Service-Preset Dispositions
|
||||
|
||||
**Issue:** #758 · **Card:** FCM-M1-003 · **Status:** M1 executable disposition evidence
|
||||
|
||||
This document records the executable disposition for every currently shipped fleet YAML artifact.
|
||||
The authoritative baseline classification remains the
|
||||
[legacy inventory](../LEGACY-EXAMPLE-PROFILE-DISPOSITION-INVENTORY.md). The executable guard is
|
||||
`packages/mosaic/src/fleet/example-profile-dispositions.ts`; its test fails if a shipped YAML
|
||||
artifact is added, removed, or left without one of the dispositions below.
|
||||
|
||||
## Disposition rules
|
||||
|
||||
- **Explicit v1 fixture:** the artifact is loaded through the existing v1 roster parser and must
|
||||
declare version: 1. It remains a compatibility fixture; it is not silently treated as a v2
|
||||
roster or given inferred aliases.
|
||||
- **Canonical profile:** the artifact is loaded through `loadProfiles`, which uses the shared
|
||||
baseline-plus-`roles.local` persona resolver and rejects unreadable or unresolved classes.
|
||||
- **Canonical service policy:** the artifact is loaded through the operator-interaction service
|
||||
policy reader and provisioned with a generic supplied identity. It validates its runtime, model,
|
||||
reasoning, and legacy tool-policy compatibility without hardcoding a product identity.
|
||||
|
||||
No artifact is retired in this card. A later retirement requires both a replacement link and a
|
||||
visible deprecation note; the executable guard must then record the new disposition before the
|
||||
artifact can be removed.
|
||||
|
||||
## Shipped artifacts
|
||||
|
||||
| Artifact | Disposition | Executable path | Compatibility notes |
|
||||
| ------------------------------------ | ------------------------ | --------------------------------- | ------------------------------------------------------------------------------------------------ |
|
||||
| `examples/coding.yaml` | Explicit v1 fixture | v1 roster parser | Retains approved `implementer` and `reviewer` compatibility inputs. |
|
||||
| `examples/general.yaml` | Explicit v1 fixture | v1 roster parser | Retains unresolved `worker` without an inferred canonical role. |
|
||||
| `examples/hybrid.yaml` | Explicit v1 fixture | v1 roster parser | Retains `implementer`, `reviewer`, and resolver-dependent `researcher`. |
|
||||
| `examples/local-canary.yaml` | Explicit v1 fixture | v1 roster parser | Retains the local-tmux canary topology. |
|
||||
| `examples/minimal.yaml` | Explicit v1 fixture | v1 roster parser | Retains `canary` without an inferred canonical role. |
|
||||
| `examples/operator-interaction.yaml` | Explicit v1 fixture | v1 roster parser | Keeps Tess only as an example instance name; `operator-interaction` remains compatibility input. |
|
||||
| `examples/research.yaml` | Explicit v1 fixture | v1 roster parser | Retains resolver-dependent `researcher` and `analyst`. |
|
||||
| `profiles/business.yaml` | Canonical profile | shared profile/persona resolver | Every referenced business class must resolve to a readable contract. |
|
||||
| `profiles/marketing.yaml` | Canonical profile | shared profile/persona resolver | Every referenced marketing class must resolve to a readable contract. |
|
||||
| `profiles/personal-assistant.yaml` | Canonical profile | shared profile/persona resolver | No interaction equivalence is inferred. |
|
||||
| `profiles/research.yaml` | Canonical profile | shared profile/persona resolver | Every research class must resolve to a readable contract. |
|
||||
| `profiles/software-delivery.yaml` | Canonical profile | shared profile/persona resolver | Retains the governance profile; authority validation remains FCM-M1-002 evidence. |
|
||||
| `services/operator-interaction.yaml` | Canonical service policy | service-policy reader/provisioner | Generic provisioning supplies the instance name; the policy itself never names Tess. |
|
||||
|
||||
## M4 migration-preview evidence
|
||||
|
||||
FCM-M4-001 layers an executable migration posture over the same 13-entry M1 inventory without
|
||||
changing the retained artifact classification:
|
||||
|
||||
- every `v1-fixture` is previewed only with explicit class and lifecycle evidence;
|
||||
- every `canonical-profile` remains validated by the shared baseline-plus-`roles.local` resolver;
|
||||
- the canonical service policy remains generic and uses only the approved tool-policy alias.
|
||||
|
||||
`validateShippedFleetMigrationDispositions` first runs the existing executable M1 guard, then requires
|
||||
explicit decisions and lifecycle observations and executes `previewV1ToV2Migration` for every shipped
|
||||
v1 fixture. `collectShippedFleetMigrationDispositions` derives the 13-entry posture directly from
|
||||
`SHIPPED_FLEET_ARTIFACT_DISPOSITIONS`, so additions or removals continue to fail the M1 guard rather
|
||||
than creating a second artifact list. None of these dispositions claims a cutover, canary, or
|
||||
rollback; those gates belong to FCM-M4-002. See [v1-to-v2 preview](./v1-to-v2.md).
|
||||
|
||||
## Running the guard
|
||||
|
||||
```fleet-command
|
||||
pnpm --filter @mosaicstack/mosaic test -- v1-v2-migration.spec.ts \
|
||||
-t "validates all 13 shipped artifacts and executes ready previews for every v1 fixture"
|
||||
```
|
||||
|
||||
The guard is intentionally limited to shipped assets and validation. It does not generate
|
||||
environment files, mutate a roster, reconcile a fleet, migrate an installed roster, or launch an
|
||||
agent.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Legacy Fleet Class Aliases
|
||||
|
||||
Fleet class compatibility is intentionally narrow. The shared resolver accepts exactly three legacy
|
||||
class names and converts them to canonical classes before persona lookup:
|
||||
|
||||
| Legacy value | Canonical value | Migration action |
|
||||
| ---------------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| `implementer` | `code` | Replace class and tool-policy references with `code`. |
|
||||
| `reviewer` | `review` | Replace class and tool-policy references with `review`. |
|
||||
| `operator-interaction` | `interaction` | Replace class and roster-v2 tool-policy references with `interaction`. The legacy service artifact remains compatible. |
|
||||
|
||||
Alias support preserves existing inputs while provisioning and typed semantic output use canonical
|
||||
identities. Requested and canonical class values remain separately observable during semantic
|
||||
validation.
|
||||
|
||||
## Lookup and override behavior
|
||||
|
||||
Canonicalization precedes baseline and `roles.local` lookup. A legacy-named override such as
|
||||
`roles.local/implementer.md` is not a separate authority and is not selected for an `implementer`
|
||||
request. Customize the canonical role instead, for example `roles.local/code.md`.
|
||||
|
||||
The compatibility file `operator-interaction.md` remains shipped, but `interaction` is the canonical
|
||||
role class. Tess is an example display name only.
|
||||
|
||||
## Unresolved and custom classes
|
||||
|
||||
No names are inferred from historical usage, instance names, or similar wording. `worker`, `analyst`,
|
||||
`canary`, Tess, and Ultron are not aliases. An otherwise unknown class is accepted only if the shared
|
||||
resolver can read an actual baseline or `roles.local` contract for that exact class. A `LIBRARY.md`
|
||||
row without a readable contract fails semantic validation.
|
||||
|
||||
Custom classes receive no protected authority implicitly. Protected class/tool-policy mismatches
|
||||
fail closed.
|
||||
|
||||
## Retirement guidance
|
||||
|
||||
New configuration should emit canonical values. Existing inputs may use the three aliases during the
|
||||
compatibility period, but operators should migrate class and tool-policy fields together. Do not
|
||||
create new legacy-named role overrides; move their intended content to the canonical filename and
|
||||
validate the roster/profile before removing the old artifact.
|
||||
@@ -0,0 +1,86 @@
|
||||
# Previewing a Fleet Roster v1-to-v2 Migration
|
||||
|
||||
**Issue:** #758 · **Card:** FCM-M4-001 · **Effect boundary:** preview only
|
||||
|
||||
mosaic fleet migrate-v1 preview inventories a v1 roster and emits a canonical v2 candidate plus
|
||||
recovery evidence. It does not write a roster, apply environment projections, invoke systemd or
|
||||
`tmux`, contact connectors or remote hosts, launch an agent, run a canary, or execute rollback.
|
||||
FCM-M4-002 owns reversible cutover and rollback.
|
||||
|
||||
## Inputs
|
||||
|
||||
```fleet-command
|
||||
mosaic fleet migrate-v1 preview \
|
||||
--source roster-v1.yaml \
|
||||
--decisions migration-decisions.json \
|
||||
--observations reviewed-observations.json
|
||||
```
|
||||
|
||||
The command emits one JSON object and exits nonzero when the preview is blocked, including when any of
|
||||
`--source`, `--decisions`, or `--observations` is omitted, passed without a path value, or passed an empty
|
||||
path value. These request-shape failures are reported before any input file is read. Decision and
|
||||
observation JSON is validated fail-closed: unknown fields, malformed values, and records for non-local
|
||||
agents are rejected. Decisions must supply a positive v2 `generation`, a reviewed `fleetHost` whenever
|
||||
v1 agents include `host` or `ssh`, explicit `defaultRuntime`, and per-local-agent provider, model,
|
||||
reasoning, enabled state, and launch policy. The v1 source remains authoritative for socket semantics:
|
||||
a supported declared socket field, including an explicit empty value for the default tmux server, is
|
||||
preserved; if both supported root aliases are absent, the production v1 default is the literal empty socket.
|
||||
A matching `socketName` decision is accepted and an incompatible decision blocks, but a decision never
|
||||
supplies or repairs a missing source socket. If v1 omitted `tool_policy`, decisions must supply an
|
||||
explicit replacement; it is never derived from `class`. `model_hint` is never split or treated as
|
||||
authority.
|
||||
|
||||
Observations are separate reviewed evidence keyed by local agent name:
|
||||
|
||||
```json
|
||||
{
|
||||
"coder0": { "systemd": "inactive", "tmux": "missing" }
|
||||
}
|
||||
```
|
||||
|
||||
Only `active` plus `present` maps to `running`; only `inactive` plus `missing` maps to `stopped`.
|
||||
Missing, extra, unknown, or contradictory evidence blocks output. An observed-running agent cannot
|
||||
be marked disabled. Observed-stopped agents always remain stopped.
|
||||
|
||||
## Field disposition
|
||||
|
||||
| v1 field | v2 disposition |
|
||||
| ------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `version`, `transport`, `tmux`, `defaults`, `runtimes` | Inventoried and structurally compiled; omitted runtimes retain v1 built-in defaults, while each explicitly declared runtime without a reset field follows the production v1 `/clear` fallback; present-empty holder/work-directory/reset values block |
|
||||
| agent `name`, `alias`, `runtime`, working directory, persona/reset flags | Copied or explicitly defaulted only when absent; present-empty alias/work-directory values block for explicit disposition. Canonical ~/~/... values stay unchanged in roster evidence and traversal-free forms expand only at the shared production environment-projection boundary before unchanged absolute-path validation |
|
||||
| `provider`, `model_hint`, `reasoning_level` | Explicit provider/model/reasoning decisions; no model-hint inference |
|
||||
| `class`, `tool_policy` | Only approved aliases canonicalize automatically; other classes require explicit preserve/replace disposition and shared-resolver validation |
|
||||
| `kickstart_template` | No v2 field; explicit inventory-only disposition required |
|
||||
| agent `host`, `ssh` | host != fleetHost is demonstrably remote and inventory-only; host == fleetHost stays local; SSH targets with or without an explicit user must agree with `host`; ssh-only, missing fleet-host evidence, or contradictory targets block |
|
||||
| agent `socket` | Same-host candidate only when it matches the canonical fleet socket; conflicts block for explicit future disposition |
|
||||
| root `connector` | Inventory-only; never contacted or reconciled |
|
||||
| unknown fields or snake/camel synonym collisions | Inventoried and block readiness |
|
||||
| `.env.generated` | Rebuild from canonical roster data |
|
||||
| no legacy `.env` | `absent`; no legacy action required |
|
||||
| legacy `.env` containing generated keys only | `regenerate-only`; replace later from canonical roster data |
|
||||
| legacy `.env` containing strict local keys | `relocate-local`; preserve those keys in `.env.local` during a later reviewed cutover |
|
||||
| legacy `.env` containing forbidden/unsafe/sensitive/malformed keys | `quarantine`; private input only, with diagnostics limited to code, key, and SHA-256 |
|
||||
|
||||
The only automatic aliases are implementer → code, reviewer → review, and
|
||||
operator-interaction → interaction. Similar or domain-specific names are never inferred. Automatic
|
||||
classes do not accept competing disposition records. Semantic validation delegates to the existing
|
||||
baseline-plus-`roles.local` resolver after the candidate is compiled by the existing v2 compiler.
|
||||
|
||||
## Evidence and recovery boundary
|
||||
|
||||
Ready output includes source and candidate SHA-256 identities, value-free field inventory, excluded
|
||||
remote/connector entries, explicit environment dispositions with sanitized diagnostics, and the lifecycle
|
||||
evidence used for each local candidate. Canonical lifecycle and remote-exclusion evidence ordering compares
|
||||
Unicode code points directly and does not depend on source-agent order or process locale. Source field
|
||||
inventory remains position-addressed evidence of the exact input. Recovery is marked non-executable and
|
||||
assigns the executable gate to FCM-M4-002.
|
||||
|
||||
Before any later cutover, preserve these artifacts:
|
||||
|
||||
1. authoritative v1 roster backup;
|
||||
2. agent environment backup, including `.env.local` and private quarantine inputs;
|
||||
3. reviewed lifecycle observations;
|
||||
4. canonical candidate v2 roster and its SHA-256.
|
||||
|
||||
See [backup and restore](../operations/backup-restore.md). Preview output is migration-readiness
|
||||
evidence, not proof that migration, canary, or rollback occurred.
|
||||
@@ -0,0 +1,429 @@
|
||||
# Mosaic Fleet — North Star
|
||||
|
||||
> **Workstream:** W-FLEET (Fleet) under mission `mvp-20260312`
|
||||
> **Umbrella:** [docs/MISSION-MANIFEST.md](../MISSION-MANIFEST.md) · [docs/PRD.md](../PRD.md) (Mosaic Stack v0.1.0)
|
||||
> **Status:** doctrine — authored 2026-06-20. Owner of this file: Fleet workstream lead.
|
||||
> This document does **not** modify the MVP rollup; a rollup row is proposed, not written here.
|
||||
|
||||
## Vision
|
||||
|
||||
A **customizable, multi-tenant fleet of always-on AI agents** — each defined by role,
|
||||
materialized as a durable, joinable runtime session, coordinated by the proven
|
||||
orchestrator/worker model, and observable end-to-end across hosts. Coding today;
|
||||
finance, analytics, research as roster entries tomorrow — same primitives, different
|
||||
roster. The fleet is the **agent-session execution layer** of the Mosaic Stack MVP:
|
||||
the thing federation makes reachable across hosts and the webUI/TUI/CLI make visible.
|
||||
|
||||
The USC tmux PoC (durable sessions + `agent-send` comms) proved the model. This
|
||||
workstream makes it an official, observable, multi-tenant Mosaic Stack capability.
|
||||
|
||||
## The Fleet as means of production (bootstrapping)
|
||||
|
||||
The Fleet has a **dual role**, and that is the point:
|
||||
|
||||
- **As product** — a multi-tenant agent-fleet capability of Mosaic Stack (this workstream).
|
||||
- **As means of production** — the orchestrator/worker fleet that _actually builds the
|
||||
entire MVP_ (federation W1, webUI, TUI, CLI, and the Fleet itself).
|
||||
|
||||
We are **building the system that builds the system.** Every other MVP workstream is
|
||||
delivered _by_ the fleet, so fleet observability and control are not merely product
|
||||
features — they are the **operational floor of the whole delivery effort**. If we cannot
|
||||
see and steer the agents, we cannot trust what they ship. This is why Phase 2
|
||||
(observability) leads: it is the instrument panel for the factory, dogfooded on the live
|
||||
fleet that is, recursively, building Mosaic Stack.
|
||||
|
||||
The discipline that makes great power safe is the same gate chain the fleet enforces:
|
||||
independent review before merge, green CI, honest completion, decide-and-inform cadence,
|
||||
and no irreversible action without authority. The bootstrap is only as trustworthy as
|
||||
those gates.
|
||||
|
||||
## Alignment with MVP cross-cutting requirements
|
||||
|
||||
The Fleet inherits — does not re-invent — the MVP's hard requirements:
|
||||
|
||||
| MVP req | What it means for the Fleet |
|
||||
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
|
||||
| MVP-X1 three-surface parity | fleet observability/control reachable via **CLI + TUI + webUI** (CLI first; webUI is required for parity, not optional) |
|
||||
| MVP-X2 multi-tenant isolation | one tenant = one **Linux uid** (own systemd --user, socket, ~/.config/mosaic); no cross-tenant leakage |
|
||||
| MVP-X3 auth (BetterAuth/SSO) | operator→fleet and cross-host views are auth-gated through the platform's existing auth |
|
||||
| MVP-X4 quality gates | pnpm typecheck/`lint`/`format:check` green before any push |
|
||||
| MVP-X5 federated topology | cross-host fleet visibility rides the **federation** boundary (W1), not a bespoke broker |
|
||||
| MVP-X6 OTEL tracing | heartbeats, sends, and lifecycle events emit spans; `traceparent` crosses the federation boundary |
|
||||
| MVP-X7 trunk merge | branch from `main`, squash-merge via PR, never push to `main` |
|
||||
|
||||
## The stack — where every concern lives
|
||||
|
||||
One **definition** is the source of truth; the **session** is how it runs.
|
||||
|
||||
| Layer | Owner | Phase-2 reality | Destination |
|
||||
| -------------------------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Definition + identity + auth** | gateway / `mosaic-as` (scoped tokens, #541) | `roster.yaml` (tenant-tagged) | one definition; mosaic agent --new materializes it |
|
||||
| **Tenancy boundary** | **Linux uid per tenant** (linger, own systemd --user, own socket, own ~/.config/mosaic) | one tenant: `jarvis` = tenant zero | uid-per-tenant; federation aggregates across hosts |
|
||||
| **Runtime** | per-tenant tmux session on isolated socket | dogfood stub sessions (live now on `mosaic-factory`) | claude/codex/pi/opencode TUIs |
|
||||
| **Liveness** | **heartbeat protocol** every runtime answers | protocol defined + dogfood stub answers it | all runtimes answer; "healthy" ≠ "pane alive" |
|
||||
| **Observation** | read-only `watch` (native tmux) + `pipe-pane` stream | CLI `watch`/`ps`; explicit opt-in `attach` for control | + auth-gated webUI streams |
|
||||
| **Control plane** | **federation** across hosts × tenants | records already carry `tenant_id` + `host` | federated gateways expose fleet state; webUI in Phase 5 |
|
||||
| **Central register** | Postgres `fleet` schema (gateway instance); access via gateway API only | _none in PoC_ (files + `roster.yaml`) | agents, missions, tasks, heartbeats, spend — single network-accessible SSOT; docs = generated projections |
|
||||
| **Budget / spend governance** | **per-tenant budget policy** ingested by the orchestrator + routing layer | none today (spend is unmetered) | usage-vs-limit feedback ingested; spend auto-paced to the limit window; per-provider/per-account/concurrency/API-$ budgets enforced |
|
||||
|
||||
> **PoC socket hygiene:** the PoC fleet runs on the **default tmux socket** (no `-L`).
|
||||
> The named production-isolation socket is **`mosaic-fleet`** (matches the product brand);
|
||||
> an absent roster `socket_name` means the default socket everywhere (spawn, fleet ps,
|
||||
> onboarding cheat-sheet). The legacy dogfood canary still runs on the old `mosaic-factory`
|
||||
> socket pending migration.
|
||||
|
||||
## Operating model (inherited, not reinvented)
|
||||
|
||||
The AI-guide law stands: one accountable **orchestrator**, isolated **workers** that
|
||||
stop at PR-open, the serialized **gate chain** (independent review → green CI →
|
||||
diff-sanity → squash-merge → verify), **decide-and-inform** cadence, and a durable
|
||||
**board** so missions survive session death. The Fleet is the infrastructure _under_
|
||||
this model. See `mosaicstack-aiguide` whitepapers 01 (inter-agent comms) and 03
|
||||
(orchestration model) for the rationale.
|
||||
|
||||
## Fleet roster — the two-agent floor and the role library
|
||||
|
||||
A fleet is **never a single agent**. The minimum viable fleet is **two**:
|
||||
|
||||
| Role | Mandate | Boundaries |
|
||||
| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
|
||||
| **Orchestrator** | The user's **single point of contact**. Owns the general flow, keeps agentic actions on-target, and **adds/removes agents from the fleet at will** to meet goals and user needs. Exactly **one** per fleet (the existing R5 invariant). | Delegates source work; never the sole worker. |
|
||||
| **Enhancer** | The fleet's **continuous-improvement loop**. Monitors fleet activity, analyzes for enhancements/optimizations, builds a **plan of remediation**, and — **with the orchestrator** — upgrades fleet capability: tool creation/repair, skills, harness improvements, and **bug reports filed to Mosaic Stack** for proper remediation. Recommends which agents are needed. | **Does not code, review code, or perform delivery tasks.** Improvement and diagnosis only. |
|
||||
|
||||
> **Why two, not one:** the orchestrator drives delivery; the enhancer makes the fleet
|
||||
> _get better at delivering_ over time. The enhancer is how the fleet self-heals its tools,
|
||||
> skills, and harnesses, and how real defects flow back to Mosaic Stack as bug reports.
|
||||
> Together they are the irreducible core — every other role is added on demand.
|
||||
|
||||
A **general** fleet starts at this floor: the orchestrator (advised by the enhancer)
|
||||
materializes whatever roles prove necessary over the mission's life. Specialized presets
|
||||
(coding, research, etc.) seed additional roles up front, but all reduce to the same two-agent
|
||||
spine plus an on-demand **role library**:
|
||||
|
||||
| Role profile | Purpose |
|
||||
| ------------------- | --------------------------------------------------------------------------------- |
|
||||
| **orchestrator** | point of contact, flow control, fleet composition (1 per fleet) |
|
||||
| **enhancer** | fleet monitoring, optimization, tool/skill/harness upgrades, upstream bug reports |
|
||||
| **coder** | implementation (worker; stops at PR-open) |
|
||||
| **code review** | independent code review gate |
|
||||
| **security review** | security/auth/secret review gate |
|
||||
| **research** | investigation, synthesis, options analysis |
|
||||
| **board** | deliberation panel — moonshot, contrarian, technical, business, financial lenses |
|
||||
| **operations** | infra, deploy, health, incident response |
|
||||
| _…extensible_ | new profiles added as missions demand (orchestrator + enhancer decide) |
|
||||
|
||||
## Invariants — "maximal vision, incremental delivery, zero foreclosure"
|
||||
|
||||
Every artifact, starting Phase 2, MUST:
|
||||
|
||||
1. Carry **`tenant_id` + `host`** in schema and message addressing — even with one of each today.
|
||||
2. Treat **isolation socket ≠ invisibility** — anything isolated is surfaced by one command.
|
||||
3. Define **healthy = answered a heartbeat within N seconds**, never just "pane alive".
|
||||
4. Make **observation read-only by default**; control is an explicit, separate, opt-in verb.
|
||||
|
||||
> **OPS INVARIANT — runtime agents need a real TTY.** Claude/Codex/pi/opencode agents
|
||||
> cannot be bare-launched from a systemd `ExecStart`; a durable harness with a real PTY is
|
||||
> required. This is **why `start-agent-session.sh` launches into tmux** and uses a
|
||||
> `MOSAIC_AGENT_COMMAND` override rather than running the runtime directly under systemd.
|
||||
|
||||
## Budget & token governance (first-class fleet concern)
|
||||
|
||||
Spend is a fleet-level resource, not a per-agent afterthought. The fleet treats token
|
||||
and API-dollar budget the way it treats liveness: a signal every runtime exposes and the
|
||||
control plane is accountable for. This rides the same primitives as everything else —
|
||||
`tenant_id` + `host` on every spend record, **read-only metering by default**, and the
|
||||
**federation** layer as the cross-host aggregation point (W1) — so budgeting is zero-foreclosure
|
||||
from day one even while one tenant exists.
|
||||
|
||||
**Two spend regimes, one policy surface:**
|
||||
|
||||
| Regime | Feedback signal | Fleet obligation |
|
||||
| ------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------- |
|
||||
| **OAuth-subscription runtimes** (Claude sub, Codex sub) | runtime exposes **current-usage-vs-limit** within a rolling limit window | **ingest** the signal per sub-account; **auto-pace** agentic spend so the window is not exhausted early |
|
||||
| **API-token runtimes** (metered per token) | provider billing / token counts | enforce **hard $-spend ceilings**; on breach, **downgrade → queue → refuse** (below) |
|
||||
|
||||
**Auto-pacing law (OAuth subs) — EVEN-SPREAD default (Jason override, 2026-06-22):** the fleet
|
||||
paces agentic token spend to consume the limit window **evenly over remaining time**:
|
||||
target rate = _(remaining usage available)_ ÷ _(remaining time in the window)_. Example: 100% of
|
||||
a 7-day window = **~14.285%/day**; the system tracks current usage and continuously re-splits the
|
||||
remainder evenly to hold pace. **Anticipated token-spend-per-task is the budgeting informant** —
|
||||
tasks are scheduled against the daily pace, not run until the quota is gone. Rationale: spreading
|
||||
delivery evenly beats rapidly exhausting usage and losing **multiple days of momentum**.
|
||||
**Rapid pacing / overspend requires EXPLICIT user authorization;** absent it, even-spread holds.
|
||||
Pacing is a control-plane decision, surfaced read-only before it throttles a lane.
|
||||
|
||||
**Hard-cap breach behavior (ladder):** when a budget ceiling is hit mid-work, the fleet
|
||||
**downgrades first** (opus → sonnet → haiku, then Claude → Codex), **queues** the lane at the
|
||||
cheapest floor until the window resets, and **refuses** only as a last resort. Refusal is never
|
||||
the first response to a breach.
|
||||
|
||||
**Spend accounting, learning & telemetry:**
|
||||
|
||||
- **Multi-subscription auto-routing:** a tenant with multiple subscriptions may let the fleet
|
||||
**auto-route work to the account with the most available usage** (within budget policy).
|
||||
- **Historical spend learning:** every task's token spend is **recorded**; historical data
|
||||
continuously updates known **spend-per-task**, **typical daily spend**, and projections — so
|
||||
estimates self-correct and pacing stays on target.
|
||||
- **Projected + actual spend on artifacts (Mosaic Stack mandate):** PRDs, missions, and task
|
||||
decomposition **MUST note projected AND actual token spend** — a Mosaic Stack process standard
|
||||
(template-level), tracked separately as **#622**.
|
||||
- **Anonymized telemetry → mosaicstack.dev:** spend data is reported (anonymous) to the
|
||||
mosaicstack.dev telemetry endpoint so other agents/fleets budget and optimize from real,
|
||||
anonymized data. Product workstream, tracked separately as **#623**.
|
||||
|
||||
**User-settable budgets (the policy surface).** A tenant operator can set budgets for every
|
||||
configured **provider** (per-provider ceilings), the **account-to-task mapping**, the **agentic
|
||||
routing flow**, **concurrency** (the spend multiplier), and **hard API-token $-limits**. Budgets
|
||||
are enforced at the orchestrator + routing boundary, not inside individual workers (a worker never
|
||||
decides its own budget — see delegation discipline).
|
||||
|
||||
**Budget CLI UX (#558):** mosaic budget set --reset-at sets the window reset; reset-datetimes
|
||||
carry **confidence tags** (`user` / `provider` / `estimated` / `unknown`); and **urgency/criticality
|
||||
is a dispatch-gate modifier** — high-urgency work may override even-spread pacing **within
|
||||
authorization**. (Also feeds the budgeting workstream, not only this doc.)
|
||||
|
||||
## Observation model
|
||||
|
||||
| Verb | Behavior |
|
||||
| --------------------------------- | -------------------------------------------------------------------------------------------------- |
|
||||
| mosaic fleet ps | one table joining systemd + tmux + process + idle + last-heartbeat, with drift + boot-enable flags |
|
||||
| mosaic agent watch <name> | **read-only** join (grouped session / `-r`), no resize tyranny, no keystrokes |
|
||||
| mosaic agent attach <name> | explicit interactive takeover (the only path that can type) |
|
||||
| mosaic agent send <name> --verify | confirms message **accepted**, not merely keystroke-injected |
|
||||
|
||||
> Why the current PoC blocks observation: sessions live on the isolated `mosaic-factory`
|
||||
> socket (invisible to default tmux ls), the only sanctioned read is `capture-pane`
|
||||
> (blank for full-screen TUIs), and `attach` is read-write + resizes the session. The
|
||||
> verbs above restore "join and observe" safely.
|
||||
|
||||
## Control plane & central register
|
||||
|
||||
### Why the register must be Postgres
|
||||
|
||||
The fleet is multi-host (w-jarvis + dragon-lin + future). A SQLite file is a local
|
||||
file — it is not a network service and cannot be shared across hosts. Beyond topology,
|
||||
Postgres MVCC eliminates the concurrent-writer corruption class Hermes hit with SQLite
|
||||
under multi-agent access.
|
||||
|
||||
Access is exclusively through the **gateway API** (`apps/gateway` — typed, auth-gated,
|
||||
scoped tokens). No agent or dispatcher pane ever holds a raw DB credential; a
|
||||
compromised pane cannot corrupt or exfiltrate the register.
|
||||
|
||||
### Architecture (layers)
|
||||
|
||||
| Layer | Responsibility | Implementation |
|
||||
| ---------------------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Register** | Source of truth: agents, missions, tasks, heartbeats, spend | Postgres `fleet` schema — existing stack instance (`@mosaicstack/db`) |
|
||||
| **Access** | Typed, auth-gated API | Gateway fleet/\* routes |
|
||||
| **Dispatcher** | Brief classification, BOD review, planning/coding/review/test/deploy sequencing + gates → fleet task dispatch | **forge pipeline engine** (`runPipeline`/`resumePipeline`, brief classifier, BOD) **+ thin `forge-exec` adapter → `agent-send.sh`**; NOT a new daemon — forge is reused, only stage→agent dispatch is new |
|
||||
| **Orchestrator (Mos)** | Goals, missions, judgment, user/PA interface | Context-light; sets intent → re-engages only for decisions |
|
||||
|
||||
### Dispatcher = forge (reuse, do not rebuild)
|
||||
|
||||
The dispatcher is **not new work**: it is `@mosaicstack/forge`, a fully-implemented
|
||||
software-factory pipeline engine (brief → Board-of-Directors review → 3 planning stages →
|
||||
coding → review/remediation → testing → deploy). Forge already provides
|
||||
`runPipeline`/`resumePipeline`, a brief classifier, and a BOD persona loader, so the fleet
|
||||
does **not** re-implement sequencing, gate logic, or brief classification. The only new
|
||||
fleet-owned code is a thin **`forge-exec` TaskExecutor adapter** (`ForgeTask` →
|
||||
`agent-send.sh` to a named agent) — forge's single missing piece — tracked as a Gitea
|
||||
issue and built post-PoC. The Postgres register backs forge's pipeline state (durable
|
||||
`resumePipeline`, cross-host) in addition to cross-project missions/tasks/Kanban. The
|
||||
north-star **'board' role IS forge's Board-of-Directors** — reused from forge, not a new
|
||||
role implementation.
|
||||
|
||||
### Docs as projections
|
||||
|
||||
`docs/TASKS.md` and `MISSION-MANIFEST.md` are **generated projections** of the DB,
|
||||
not hand-maintained. The dispatcher (or a scheduled job) renders Markdown from
|
||||
fleet.\* tables and commits the output. DB is authoritative; docs are for human
|
||||
reference.
|
||||
|
||||
### Spend
|
||||
|
||||
`fleet.spend_ledger` records projected and actual token spend per agent/mission/task
|
||||
(ties to issue #622). The dispatcher enforces budget caps before dispatching. Mos reads
|
||||
the roll-up via API — no raw DB access, no context-bloating dumps.
|
||||
|
||||
### Federation
|
||||
|
||||
Cross-host fleet state flows through federated gateway queries (existing
|
||||
`federation_peers` / `federation_grants` machinery). This is the existing north-star
|
||||
invariant: **control plane rides federation (W1), not a bespoke broker.** No new
|
||||
broker introduced.
|
||||
|
||||
### Scope
|
||||
|
||||
This is Phase 4–5 of this roadmap, materialized. It MUST NOT block the PoC (which
|
||||
runs correctly on files + `roster.yaml`). Begin when Phase 2 heartbeat protocol is
|
||||
stable and concurrent-agent count makes file coordination the bottleneck.
|
||||
|
||||
### Open sub-decision
|
||||
|
||||
Dedicated Postgres **instance** vs. dedicated **schema** in the existing instance.
|
||||
Recommendation: dedicated schema, existing instance (a migration file, not new infra);
|
||||
re-evaluate if isolation or write-volume demands it.
|
||||
|
||||
## Phased roadmap
|
||||
|
||||
| Phase | Outcome | Status |
|
||||
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
|
||||
| 0–1 | tmux PoC, hardening, published CLI v0.0.34 (#565–#568) | ✅ done |
|
||||
| **2 — Observability** | fleet ps (host+tenant aware join), heartbeat protocol + dogfood stub answers it, agent watch (read-only), agent send --verify receipts | ▶ now |
|
||||
| 3 — Real runtimes | claude/codex/pi/opencode answer heartbeat; **hybrid lifecycle** (core always-on: **orchestrator + enhancer**; ephemeral workers per lane) | planned |
|
||||
| 4 — Unified definition | one agent schema in gateway; mosaic agent --new → materialized per-tenant session; uid-tenant provisioning; **`fleet` schema migration + `forge-exec` TaskExecutor adapter (forge → `agent-send.sh`)** | planned |
|
||||
| 5 — Control plane | federation-backed cross-host × cross-tenant fleet view; **webUI** (surface chosen then) for MVP-X1 parity; **central register live (spend ledger, docs-as-projections, multi-host Kanban)** | planned |
|
||||
|
||||
## Decisions of record (2026-06-20, with Jason)
|
||||
|
||||
- Agent model: **config defines, session runs** (gateway = definition/identity/auth; tmux = runtime).
|
||||
- Tenancy: **multi-tenant from the start**; isolation = **per-tenant Linux uid**.
|
||||
- Health: **heartbeat required** (dogfood stub implements the protocol now).
|
||||
- Lifecycle: **hybrid** — core always-on + ephemeral workers per lane.
|
||||
- Observation: **read-only default, opt-in takeover**.
|
||||
- Multi-host: **designed-for from day one**; control plane **rides federation (W1)**.
|
||||
- Delivery: **CLI-first now**, dogfood against the live stub fleet; webUI deferred to Phase 5.
|
||||
- Runtimes: fleet agents default to **Codex / pi-on-Codex**; **Claude is reserved for Claude
|
||||
Code only** (avoid alternate-harness API pricing). Validated durable recipe:
|
||||
mosaic yolo pi --model openai-codex/gpt-5.5:high. Durable detached launch requires the
|
||||
runtime-bin on PATH (baked into the pane command) + boot-survival (`enable` + linger),
|
||||
which fleet init should automate.
|
||||
|
||||
## Decisions of record (2026-06-22, with Jason)
|
||||
|
||||
- **Two-agent floor:** every fleet has, at minimum, an **orchestrator** and an **enhancer**.
|
||||
The orchestrator is the user's point of contact and composes the fleet; the enhancer runs the
|
||||
continuous-improvement loop (monitor → analyze → remediate → upgrade tools/skills/harness →
|
||||
file Mosaic Stack bug reports) and **does not code or review**.
|
||||
- **Role library:** orchestrator, enhancer, coder, code review, security review, research,
|
||||
board (moonshot/contrarian/technical/business/financial), operations — extensible; the
|
||||
orchestrator (advised by the enhancer) adds roles as missions demand.
|
||||
- **Orchestrator chat connector:** the orchestrator is reachable over a user-chosen connector
|
||||
(tmux now; Telegram/Discord/Matrix/Slack configurable). Validated live: **"Mos" orchestrator
|
||||
on Discord** via the Claude Code discord channel plugin (w-jarvis).
|
||||
- **Session context cap = 200k tokens (GLOBAL to all Claude sessions):** Claude Code sessions are
|
||||
capped at a **max 200k-token context window**. Long-running sessions extended toward 1M tokens
|
||||
have proven **worse in practice** (degraded steering, off-plan divergence); 200k is the standard.
|
||||
**Enforcement split:** the _window_ lives in **~/.claude/settings.json** (host-global) as
|
||||
"autoCompactWindow": 200000 + "autoCompactEnabled": true; the _1M-disable_ lives in **launch
|
||||
ENV** (`CLAUDE_CODE_DISABLE_1M_CONTEXT=1`, plus `CLAUDE_CODE_AUTO_COMPACT_WINDOW=200000`) wherever
|
||||
a [1m] model can be selected (`mos-claude.service` + the fleet Claude launcher), so every Claude
|
||||
agent is capped at spawn. (settings = window; env = 1M-disable.)
|
||||
- **Worker context bound (#8):** workers are kept context-bounded via the **ephemeral-per-lane
|
||||
lifecycle + native compaction**, not via the 200k knob. The explicit `autoCompactWindow` 200k knob
|
||||
**stays Claude-specific** — the _principle_ (bounded context) extends to workers, the _knob_ does not.
|
||||
- **Orchestrator delegation discipline:** the orchestrator **delegates all delivery work** to
|
||||
subagents / workflows / ultracode / coder agents and confines its own context to the personal-assistant
|
||||
lane. Keeping delivery out of the orchestrator's window keeps its context unpolluted and measurably
|
||||
reduces off-plan divergence. The orchestrator coordinates and decides; it does not implement.
|
||||
- **Budget governance is fleet doctrine:** token/API-dollar budgeting is a first-class fleet concern
|
||||
(see "Budget & token governance"). OAuth-sub usage-vs-limit feedback is ingested per account, spend
|
||||
is **auto-paced EVEN-SPREAD over remaining time** (rapid/overspend only on explicit authorization),
|
||||
spend is **tracked historically** to self-correct per-task/daily estimates, multi-sub tenants may
|
||||
**auto-route by available usage**, and operators set budgets per provider, per account-to-task
|
||||
mapping, per routing flow, per concurrency level, and as hard API-$ ceilings.
|
||||
- **Spend accounting is a Mosaic Stack process mandate:** PRDs, missions, and task decomposition
|
||||
**MUST carry projected + actual token spend**; used locally for pacing and reported as **anonymized
|
||||
telemetry to mosaicstack.dev**. The template standard (#622) and telemetry product (#623) are
|
||||
tracked separately.
|
||||
- **Unified identity = "Fleet" (Jason, 2026-06-22):** the product is **Mosaic Fleet** — one unified
|
||||
user-facing identity and CLI surface. **forge** is the Fleet's **internal** delivery/orchestration
|
||||
engine (not a separate product); the control-plane **Postgres register is the Fleet's register**;
|
||||
workers/runtime are the **Fleet substrate**. **"factory" is RETIRED as a product term** — it was
|
||||
only ever the software-factory concept (which forge implements) and the old `mosaic-factory` tmux
|
||||
socket name. The production-isolation socket is now **`mosaic-fleet`** (matches the product brand);
|
||||
the legacy dogfood canary remains on the old `mosaic-factory` socket pending migration. **Code stays
|
||||
layered** (forge + fleet + control-plane as internal layers);
|
||||
only the **identity + CLI surface unify under Fleet.**
|
||||
- **Role-based session naming (Jason, 2026-06-22):** agent tmux sessions are named by **role**
|
||||
(`orchestrator`, `enhancer`, `research`, `coder0-0`, …), not by persona. **Persona lives in
|
||||
`SOUL.md`**; the front-end / Discord presents a **friendly alias** (e.g. "Mos" = the orchestrator's
|
||||
alias). The session name is the stable addressing handle; the alias is presentation.
|
||||
|
||||
### Control plane & central register
|
||||
|
||||
- **Store:** Postgres (existing stack instance, dedicated `fleet` schema via `@mosaicstack/db`). SQLite rejected: (1) it is a local file — structurally incompatible with a multi-host fleet; (2) concurrent multi-agent writes caused repeated corruption in Hermes. "SQLite + access service" rejected as reinventing a DB server badly; "LLM agent gating DB access" rejected as slow, expensive, and a single point of failure.
|
||||
- **Access:** gateway API only (`apps/gateway`, fleet/\* routes). No raw DB credentials in any agent/dispatcher pane — directly mitigates the tmux attack-surface concern.
|
||||
- **Dispatcher = forge (reuse, not a new build):** the dispatcher IS `@mosaicstack/forge`'s pipeline engine (`runPipeline`/`resumePipeline` + brief classifier + BOD persona loader), a fully-implemented software-factory pipeline (brief → BOD review → 3 planning stages → coding → review/remediation → testing → deploy). We do **not** design/build a new dispatcher and do **not** re-implement sequencing, gate logic, or brief classification. The only new fleet-owned piece is a thin **`forge-exec` TaskExecutor adapter** (suggested package `packages/forge-exec`) mapping a `ForgeTask` → `agent-send.sh` dispatch to a named fleet agent — forge's single missing piece. It is tracked as a Gitea issue and built **post-PoC** (not now).
|
||||
- **Register backs forge:** the Postgres `fleet` register is genuinely new (neither forge nor the fleet has cross-project state). It BACKS forge's pipeline state (durable `resumePipeline`, cross-host) plus cross-project missions/tasks/Kanban.
|
||||
- **'board' role = forge BOD:** the north-star role-library 'board' role IS forge's Board-of-Directors — reused, not reinvented.
|
||||
- **Orchestration vs. dispatch:** Orchestrator (Mos) sets intent and handles judgment; forge works the mechanical pipeline (sequencing, gates, status transitions, spend ledger). LLM escalation reserved for judgment: mission decomposition, re-planning on failure.
|
||||
- **Spend in the register:** `fleet.spend_ledger` tracks projected vs. actual tokens per agent/mission/task; ties to issue #622.
|
||||
- **Docs as projections:** `docs/TASKS.md` and `MISSION-MANIFEST.md` become generated exports of the DB, not hand-maintained.
|
||||
- **Sub-decision pending:** dedicated schema in existing PG instance (recommended) vs. dedicated PG instance. Revisit if isolation or write-volume demands it.
|
||||
|
||||
## Decisions of record (2026-06-24, with Jason)
|
||||
|
||||
- **Per-agent model switch (operator-configurable, NOT a global lock):** model selection is
|
||||
**per-agent**, never a host-global pin. Claude sessions MUST NOT be locked to a single model in
|
||||
~/.claude/settings.json; each agent chooses its model independently. The plumbing already exists —
|
||||
roster `model_hint` → `MOSAIC_AGENT_MODEL` → `start-agent-session.sh` appends --model <hint> to that
|
||||
agent's harness (claude or pi); settable today via mosaic fleet add|edit <agent> --model <hint>.
|
||||
**North-star target:** surface this as a **per-agent model switch in the webUI** (with CLI/TUI parity
|
||||
per MVP-X1) — read the roster, expose a per-agent model dropdown, write `model_hint` back, and restart
|
||||
that one agent to apply. Unset = inherit the harness default. This **composes with** the budget
|
||||
downgrade ladder (opus → sonnet → haiku, then Claude → Codex): the operator sets the per-agent model
|
||||
_intent/ceiling_; budget pacing may downgrade within policy. Tracked as a Fleet `TASKS.md` entry under
|
||||
the Phase-5 webUI surface.
|
||||
- **Orchestrator runtime (confirmed live):** the **orchestrator and enhancer run Claude Opus 4.8 in the
|
||||
Claude Code harness**; only workers (coder/reviewer) run pi/gpt-5.5. Consistent with the 2026-06-20
|
||||
"Claude reserved for Claude Code only" decision (the orchestrator runs _in_ Claude Code, not an
|
||||
alternate Claude harness). Pi/gpt-5.5 as the orchestrator is permitted **only if proven** at least as
|
||||
satisfactory; absent that proof, the orchestrator stays on Claude Opus 4.8.
|
||||
|
||||
## Future enhancements (north-star, post-MVP — not on the MVP track)
|
||||
|
||||
- **Mosaic Claude Discord Plugin** — a first-party Mosaic Discord connector that properly
|
||||
implements the basic Discord functions **and native Discord threads**. Threads let a user
|
||||
separate conversation topics with the orchestrator (the pattern proven by the Hermes agent).
|
||||
A major enhancement over the current third-party channel plugin; **not required for the MVP**,
|
||||
but a committed north-star target. `ASSUMPTION:` ships as a Mosaic-owned plugin so the fleet
|
||||
controls Discord UX (threads, reactions, attachments, per-thread context) end-to-end.
|
||||
- **Matrix on a local homeserver — strategic future transport.** **F4 (in progress) IS the Matrix
|
||||
connector**: an orchestrator chat connector speaking the Matrix client-server API against a
|
||||
self-hosted homeserver (Conduit default, Synapse alt). Matrix is named here as the strategic
|
||||
future transport — peer to tmux/Discord, not superseded by them.
|
||||
- **tmux fleet attack-surface hardening.** Many always-on tmux sessions are an attack surface;
|
||||
tmux send-keys / socket access could enable malicious action against agents directly.
|
||||
Mitigations to build toward: socket ownership/perms, per-tenant socket isolation (already an
|
||||
invariant), authenticated `agent-send`, and an audit of who can write to any pane. **Post-MVP
|
||||
unless a P0 surfaces.** The control-plane register reinforces this (gateway-API access = no raw
|
||||
DB creds in panes). A not-started risk-assessment + mitigation-plan task rides the Fleet `TASKS.md`.
|
||||
|
||||
## Assumptions (veto-able)
|
||||
|
||||
- `ASSUMPTION:` first-class runtimes = claude, codex, pi, opencode; a "role" (analyst,
|
||||
finance, researcher) = persona + skills + tools on top of a runtime, shipped as a
|
||||
starter role library in the framework.
|
||||
- `ASSUMPTION:` the cross-host control plane is the **federation** layer (W1), not a
|
||||
separate `fleetd` daemon.
|
||||
- `ASSUMPTION:` Fleet is workstream **W-FLEET** under `mvp-20260312`; a rollup row in
|
||||
`docs/TASKS.md` and a workstream declaration in `MISSION-MANIFEST.md` are proposed to
|
||||
the MVP orchestrator, not written by this workstream.
|
||||
- `ASSUMPTION:` OAuth-subscription runtimes (Claude sub, Codex sub) expose a machine-readable
|
||||
current-usage-vs-limit signal the fleet can poll/ingest; if a provider exposes no such signal,
|
||||
that provider's accounts fall back to API-style hard-ceiling budgeting only (no auto-pacing).
|
||||
- `ASSUMPTION:` budget policy lives at the orchestrator + routing layer and is surfaced through the
|
||||
same CLI→TUI→webUI parity (MVP-X1) as the rest of fleet state — not a separate budgeting daemon.
|
||||
- `ASSUMPTION:` the 200k session cap is enforced by Claude Code settings/env composition (model
|
||||
variant + `autoCompactWindow`), not by a Mosaic wrapper; a wrapper is the fallback only if the
|
||||
harness later removes those knobs.
|
||||
- `ASSUMPTION:` The central register (Postgres `fleet` schema + gateway API + forge as dispatcher) is
|
||||
the Phase 4–5 control plane, begun after Phase 2 observability is proven. It is a dedicated
|
||||
**W-FLEET** sub-workstream entry, not a separate mission. The dispatcher is `@mosaicstack/forge`
|
||||
(reused, not a new daemon); the only new fleet-owned code is the thin **`forge-exec` TaskExecutor
|
||||
adapter** (suggested package `packages/forge-exec`, `ForgeTask` → `agent-send.sh`), tracked as a
|
||||
Gitea issue and built post-PoC.
|
||||
|
||||
---
|
||||
|
||||
> **Release procedure (drift re-capture, 2026-06-22):** mosaic update only propagates new fleet
|
||||
> commands when the **CLI version is bumped** — without a version bump, fleet command changes never
|
||||
> reach installed hosts. The release/version-bump procedure (bump → publish → mosaic update
|
||||
> [→ `--relaunch`]) must be documented so fleet changes actually land. (Also feeds the budgeting
|
||||
> workstream.)
|
||||
>
|
||||
> **Tracked separately (not in scope for this doc PR):** **#622** PRD/mission/task projected+actual
|
||||
> spend template standard · **#623** anonymized spend telemetry → mosaicstack.dev (product) ·
|
||||
> **#625** `tenant_id` roster-schema field (multi-tenant; invariant #1 home) · **#628** `forge-exec`
|
||||
> TaskExecutor adapter (post-PoC). This PR records **doctrine only** — no implementation.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Fleet Configuration Backup and Restore Boundary
|
||||
|
||||
**Issue:** #758 · **Card:** FCM-M4-001
|
||||
|
||||
This page defines evidence that must exist before a roster v1-to-v2 cutover. FCM-M4-001 lists these
|
||||
prerequisites in non-executable recovery evidence but does not validate that backups exist and performs
|
||||
no backup, migration, canary, or restore. FCM-M4-002 owns the executable reversible canary and rollback
|
||||
gates.
|
||||
|
||||
## Preserve before cutover
|
||||
|
||||
- The authoritative v1 roster, byte-for-byte, with a SHA-256 identity.
|
||||
- Existing per-agent legacy `.env`, strict `.env.local`, and quarantine files under private
|
||||
permissions.
|
||||
- Reviewed per-local-agent systemd and exact-socket tmux observations.
|
||||
- The canonical v2 candidate and its SHA-256 identity.
|
||||
- Inventory-only remote agents and connector configuration as evidence, not local control-plane input.
|
||||
|
||||
`.env.generated` is a rebuildable projection and is not restored as authority. It must be regenerated
|
||||
from the selected authoritative roster. `.env.local` is operator-owned strict data and must not be
|
||||
overwritten or absorbed into generated output. Quarantined source remains private evidence; public
|
||||
diagnostics expose only rule code, key name, and SHA-256.
|
||||
|
||||
## Restore requirements
|
||||
|
||||
A later rollback implementation must restore the authoritative roster and operator-owned environment
|
||||
files, regenerate managed projections, and preserve each reviewed pre-cutover stopped/running state.
|
||||
It must never start an agent observed stopped and must never reconcile an inventory-only remote or
|
||||
connector entry.
|
||||
|
||||
The preview evidence deliberately records:
|
||||
|
||||
- executable: false;
|
||||
- required backup artifacts;
|
||||
- source and candidate identities;
|
||||
- lifecycle observations and resulting desired states;
|
||||
- environment relocation/quarantine dispositions;
|
||||
- FCM-M4-002 as the executable rollback gate owner.
|
||||
|
||||
Do not interpret a ready preview as a completed backup, migration, canary, or rollback.
|
||||
@@ -0,0 +1,20 @@
|
||||
# Environment Quarantine Operations
|
||||
|
||||
Legacy <name>.env is input evidence, never current launch authority. Projection preparation classifies it deterministically:
|
||||
|
||||
- generated roster keys → discard and regenerate;
|
||||
- allowed strict local keys → relocate to private `.env.local`;
|
||||
- malformed, duplicate, unknown, sensitive-looking, shell-bearing, unsafe, or command-override entries → move the legacy input to private `.env.quarantine`.
|
||||
|
||||
## Safe response
|
||||
|
||||
1. Stop and read the stable error code and reported key name/hash. Do not request or paste the value.
|
||||
2. Confirm the canonical roster contains the intended non-sensitive desired state.
|
||||
3. If the key is an allowed local machine-data field, place only its validated data form in `.env.local` under private permissions.
|
||||
4. Remove unsupported intent rather than translating it into commands, channels, secret references, or unknown MOSAIC*AGENT*\* keys.
|
||||
5. Regenerate `.env.generated` from the roster and rerun a dry-run/verification gate.
|
||||
6. Retain quarantine evidence privately until the operator's normal retention process permits removal.
|
||||
|
||||
The launcher never reads quarantine. Public/JSON diagnostics expose stable code, key name where safe, and SHA-256 only—never a legacy sensitive value, credential, rejected command, or full line. Quarantine does not prove remediation, backup, migration, or rollback.
|
||||
|
||||
See [generated launch chain](../concepts/generated-env-launch-chain.md), [generated environment boundary](../reference/generated-env-boundary.md), and [migration field disposition](../migration/v1-to-v2.md#field-disposition).
|
||||
@@ -0,0 +1,28 @@
|
||||
# Reconcile and Recover a Local Fleet
|
||||
|
||||
## Safe sequence
|
||||
|
||||
1. Read mosaic fleet doctor and mosaic fleet status.
|
||||
2. Run mosaic fleet apply --expected-generation <n> --dry-run.
|
||||
3. Resolve stale generation, ownership mismatch, unsafe path, projection validation, or unmanaged-session findings before applying.
|
||||
4. Run mosaic fleet apply --expected-generation <n> only after the plan is understood.
|
||||
|
||||
This is per-generation convergence, not a rolling canary. Executable canary cutover/rollback remains held for FCM-M4-002; rolling local release evidence remains FCM-M5-002. Do not approximate either with repeated live apply commands.
|
||||
|
||||
The reconciler uses the exact roster tmux socket, exact holder session, private installation holder identity, and the complete expected global environment. For mutations it acquires its exclusive lock before rereading the canonical roster and fencing its generation; only that under-lock roster drives validation, planning, projections, and lifecycle effects. Before effects, its exclusive lock proves real private `MOSAIC_HOME` and `fleet` ancestors, uses a private `0600` lock leaf, and binds cleanup to the created file identity and ownership token. A fake holder, contaminated global environment, missing identity, unsafe lock path, or unmanaged session fails closed. It does not adopt, kill, or rename any unproven session. A crash can leave a stale lock for explicit operator inspection; reconciliation deliberately does not guess ownership or remove it.
|
||||
|
||||
## Partial results
|
||||
|
||||
The roster is never changed by reconciliation. If derived projection application partially fails, JSON reports:
|
||||
|
||||
```json
|
||||
{
|
||||
"applied": false,
|
||||
"authoritativeRoster": "unchanged",
|
||||
"projections": "incomplete",
|
||||
"lifecycle": "not-applied",
|
||||
"recovery": { "code": "projection-apply-failed", "action": "regenerate-projections-from-roster" }
|
||||
}
|
||||
```
|
||||
|
||||
If projections completed but lifecycle work failed, JSON reports projections: "complete", lifecycle: "incomplete", and the bounded action `rerun-after-inspecting-owned-resources`. If lock cleanup cannot be proven after an effect result, it adds cleanup: { "code": "lock-cleanup-failed", "action": "inspect-lock-before-retry" } without changing the known projection, lifecycle, or primary recovery truth. Inspect the retained lock before retrying; no rollback, release, or stale-lock removal is implied. Results do not include environment values, secrets, or privileged command content.
|
||||
@@ -0,0 +1,24 @@
|
||||
# Systemd and tmux Troubleshooting
|
||||
|
||||
Start with read-only mosaic fleet status, `doctor`, and `verify`. Do not manually adopt, rename, terminate, or recreate sessions while ownership is ambiguous.
|
||||
|
||||
## Decision table
|
||||
|
||||
| Finding | Meaning | Safe next step |
|
||||
| ------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------- |
|
||||
| Empty roster `tmux.socket_name` | Literal default tmux server | Do not substitute the named `mosaic-fleet` socket. Use roster-derived commands only. |
|
||||
| Non-empty socket | Exact named socket | Never target another socket or infer a per-agent socket. |
|
||||
| holder: missing | Required exact holder absent | Inspect installation/projection readiness; do not create an unproven holder manually. |
|
||||
| `ownership-mismatch` | Holder identity or global environment differs | Stop. Verify private install identity and managed paths before retry. |
|
||||
| `missing-session` | Desired-running roster agent lacks exact session | Check service/runtime preconditions; review apply dry-run. |
|
||||
| `unexpected-session` | Desired-stopped roster agent still has exact session | Confirm ownership; only reconciler may target the exact proven roster member. |
|
||||
| `disabled-running` | Disabled roster member is observed running | Inspect and reconcile only after ownership proof. |
|
||||
| `unmanagedSessions` | Unknown session exists on configured named socket | Report and investigate separately. Reconciler will not kill or adopt it. |
|
||||
| stale/concurrent generation | Desired state changed since plan | Reload roster/generation and recompute the plan. |
|
||||
| stale or ambiguous lock | Prior writer/cleanup cannot be proven | Inspect ownership; do not blindly remove the lock. |
|
||||
| projection failure | Derived files incomplete | Keep roster as authority and regenerate projections. |
|
||||
| lifecycle failure | Projections complete, runtime convergence incomplete | Inspect the exact owned resource, then rerun with current generation. |
|
||||
|
||||
Systemd state, tmux state, heartbeat, and generated files are observations/projections, not alternate desired state. Explicit apply/reconcile honors stopped/disabled intent, but current unit enablement and launcher projections do not yet prove lifecycle-safe reboot; inspect unit enablement before reboot and treat stopped/disabled boot preservation as an FCM-M3-002 hold. Current roster-v2 status commands also do not read heartbeat files. Executable gates do not provide site cutover/rollback or package asset-revision repair.
|
||||
|
||||
Errors and troubleshooting output never print legacy sensitive values, credential contents, or privileged command text. Use stable codes, key names/hashes, exact roster identities, and bounded recovery actions. See [status and drift](../reference/status-and-drift.md) and [reconcile and recover](reconcile-and-recover.md).
|
||||
@@ -0,0 +1,18 @@
|
||||
# Upgrade and Installed-Asset Drift
|
||||
|
||||
Fleet source assets and installed assets can differ after an update, but FCM-M5-001 does not add a trustworthy source-versus-installed revision detector or refresh command. Do not infer freshness from checkout presence, timestamps, generated environment files, running sessions, or a ready migration preview.
|
||||
|
||||
## Current safe boundary
|
||||
|
||||
- The canonical roster remains authority and must survive package/framework refresh.
|
||||
- Generated projections are rebuilt from that roster after the installed contract is independently verified.
|
||||
- Operator `roles.local`, `.env.local`, and private quarantine evidence are not generated assets and must not be overwritten.
|
||||
- Baseline roles, schemas, examples, service presets, launcher helpers, and systemd templates must move as one reviewed release set.
|
||||
- Remote/connector inventory and `mos-comms` are not promoted into permanent architecture by an update.
|
||||
- No update may start an agent persisted stopped, adopt an unmanaged session, or bypass generation/ownership checks.
|
||||
|
||||
## Explicit hold
|
||||
|
||||
FCM-M5-002 owns deterministic asset-drift checks, safe package/update refresh evidence, rolling local canary, independent validation certificate, and release evidence. Until that card lands, this page is an operational hold rather than an executable procedure: use the repository/release review path, preserve backups, and do not claim source/installed parity without exact revision evidence from the future validator.
|
||||
|
||||
See [approved deferrals](../../reports/deferred/758-fleet-config-deferrals.md) and [backup/restore boundary](backup-restore.md).
|
||||
@@ -0,0 +1,43 @@
|
||||
# Local Fleet Agent Mutations
|
||||
|
||||
FCM-M2-002 provides local roster-v2 create, get, update, delete, and plan operations. They only change desired state and derived environment projections. They never start, stop, inspect, reconcile, or otherwise act on runtimes, systemd units, tmux sessions, or heartbeats.
|
||||
|
||||
## CLI contract
|
||||
|
||||
The commands operate only on the canonical <mosaic-home>/fleet/roster.yaml v2 authority and print one JSON object to stdout. `--agent` is a JSON object with the roster agent fields expressed as `className`, `toolPolicy`, `workingDirectory`, `persistentPersona`, `resetBetweenTasks`, and launch: { "yolo": boolean }.
|
||||
|
||||
```fleet-synopsis
|
||||
mosaic fleet get <name>
|
||||
mosaic fleet plan <create|update|delete> [<name>] --expected-generation <n> [--agent '<json>'] [--persisted-start]
|
||||
mosaic fleet create --expected-generation <n> --agent '<json>' [--dry-run] [--persisted-start]
|
||||
mosaic fleet update <name> --expected-generation <n> --agent '<json>' [--dry-run]
|
||||
mosaic fleet delete <name> --expected-generation <n> [--dry-run]
|
||||
```
|
||||
|
||||
`get` returns the authoritative generation and the selected agent. plan create derives its name from `--agent`; plan update <name> and plan delete <name> require the target name. `--agent` accepts only the documented roster-v2 request fields and `launch.yolo`; unknown keys such as commands, channels, or secret references are rejected. Rejection diagnostics return only the stable `invalid-request` code and never echo a rejected value. `plan` and `--dry-run` validate the complete proposed roster and projections but write neither the roster nor projections. `--persisted-start` is available only for a create request: it records desired_state: running, but does not start a process. Without it, create records enabled: true and desired_state: stopped. Handled failures return JSON with `error.code` and exit non-zero; unclassified validation/projection failures use the redacted `mutation-failed` code.
|
||||
|
||||
## Generation, validation, and idempotency
|
||||
|
||||
Each create, update, or delete request includes `expectedGeneration`. A request whose expected value differs from the authoritative roster generation fails with `stale-generation`; reload and retry with a newly computed plan. A private mutation lock rejects concurrent writers with `concurrent-mutation`.
|
||||
|
||||
`planFleetAgentMutation` is deterministic and side-effect free. `executeFleetAgentMutation` validates the complete proposed roster through the existing structural and shared persona resolver, prepares generated/local/quarantine projections, and writes the roster authority atomically before applying derived projections. Equivalent create retries and delete requests for an already-absent agent are idempotent no-ops.
|
||||
|
||||
Delete removes only the exact <name>.env.generated projection for the removed roster entry. Operator-owned <name>.env.local, legacy <name>.env, quarantine records, and unrelated projections remain untouched. An already-absent generated projection is treated as stale derived state, not as a failed mutation.
|
||||
|
||||
## Result and recovery
|
||||
|
||||
Mutation results are JSON-safe objects with `applied`, `authoritativeRoster`, `projections`, `plan`, and—only if a derived projection write fails after the authoritative roster write—a recovery object. `applied` is true only when every roster and derived-projection write completed. The explicit state fields prevent a partial result from being mistaken for a rollback or a no-op:
|
||||
|
||||
```json
|
||||
{
|
||||
"applied": false,
|
||||
"authoritativeRoster": "committed",
|
||||
"projections": "incomplete",
|
||||
"recovery": {
|
||||
"code": "projection-apply-failed",
|
||||
"action": "regenerate-projections-from-roster"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Dry-runs and idempotent no-ops report authoritativeRoster: "unchanged" and projections: "not-applied"; a complete mutation reports "committed" and "complete". Recovery output identifies the authoritative roster path and regeneration action only. It never contains generated/local/quarantine values, credentials, or command text. A recovery result exits non-zero because the authoritative roster was persisted but derived projections require regeneration. Regenerate projections from the roster before attempting another mutation.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Fleet Control-Plane CLI
|
||||
|
||||
The local desired-state surface is mosaic fleet. It is distinct from the gateway-backed mosaic agent catalog and from legacy compatibility commands that act on roster v1.
|
||||
|
||||
## Roster-v2 desired-state commands
|
||||
|
||||
| Command | Effect | Generation | Output |
|
||||
| ---------------------------------------------------------- | ---------------------------------------------- | ---------- | -------------------------- |
|
||||
| mosaic fleet get <name> | Read one authoritative agent | no | One JSON object |
|
||||
| mosaic fleet plan <create\|update\|delete> ... | Validate proposed CRUD and projections | required | One JSON object; no writes |
|
||||
| mosaic fleet create ... [--dry-run] [--persisted-start] | Add desired state; default enabled/stopped | required | One JSON object |
|
||||
| mosaic fleet update <name> ... [--dry-run] | Replace mutable agent fields | required | One JSON object |
|
||||
| mosaic fleet delete <name> ... [--dry-run] | Remove roster member/generated projection | required | One JSON object |
|
||||
| mosaic fleet apply ... [--dry-run] | Plan or converge projections/lifecycle | required | One JSON object |
|
||||
| mosaic fleet reconcile ... [--dry-run] | Alias of the same convergence contract | required | One JSON object |
|
||||
| mosaic fleet start\|stop\|restart [<name>] ... [--dry-run] | Exact one-shot lifecycle action | required | One JSON object |
|
||||
| mosaic fleet status [<name>] | Observe desired/managed/runtime state | no | One JSON object |
|
||||
| mosaic fleet verify | Strict observational drift/ownership gate | no | One JSON object |
|
||||
| mosaic fleet doctor | Classify local drift and recovery context | no | One JSON object |
|
||||
| mosaic fleet migrate-v1 preview ... | Non-mutating field-complete migration evidence | no | One JSON object |
|
||||
|
||||
CRUD syntax and full payload shape are documented in [agent mutations](agent-mutations.md). Reconciliation syntax:
|
||||
|
||||
```fleet-synopsis
|
||||
mosaic fleet apply --expected-generation <n>
|
||||
mosaic fleet reconcile --expected-generation <n>
|
||||
mosaic fleet start [<name>] --expected-generation <n> [--dry-run]
|
||||
mosaic fleet stop [<name>] --expected-generation <n> [--dry-run]
|
||||
mosaic fleet restart [<name>] --expected-generation <n> [--dry-run]
|
||||
mosaic fleet status [<name>]
|
||||
mosaic fleet verify
|
||||
mosaic fleet doctor
|
||||
mosaic fleet migrate-v1 preview --source <path> --decisions <path> --observations <path>
|
||||
```
|
||||
|
||||
`get` is the read/show operation for one v2 agent. Full roster parsing and semantic validation occur on every v2 mutation/reconcile path; there is no separate mutable “config store.” The executable JSON Schema and validated example provide offline structural evidence. The PRD requires an explicit programmatic mosaic fleet validate operation, but the current CLI does not expose one; do not substitute another command or claim that requirement is delivered. This remains an implementation gap for #758.
|
||||
|
||||
## JSON and exit behavior
|
||||
|
||||
Roster-v2 CRUD and reconciler precondition failures emit { "error": { "code": "..." } } and exit non-zero. Migration preview has its own result envelope: a non-ready preview emits { "status": "blocked", "blockers": [...] } and exits non-zero rather than using the CRUD/reconciler error object. Use both exit status and command-specific state fields. A partial reconciliation result distinguishes `authoritativeRoster`, `projections`, `lifecycle`, `recovery`, and optional `cleanup`; it never claims automatic rollback. `verify` exits non-zero for drift, ownership failure, or unmanaged sessions. Sensitive legacy values, credentials, and rejected command text are never printed.
|
||||
|
||||
## Compatibility and scope
|
||||
|
||||
Roster-v1 initialization, provisioning, profiles/personas, and historical fleet add/remove remain compatibility surfaces, not roster-v2 CRUD aliases. New v2 automation should use the table above. migrate-v1 preview writes nothing and has no cutover, canary, or rollback option.
|
||||
|
||||
mosaic agent is a separate catalog/transport surface; it does not own <MOSAIC_HOME>/fleet/roster.yaml desired state. Remote/SSH reconciliation, connector mutation, arbitrary commands/channels, secret references, and gateway convergence are rejected or outside scope.
|
||||
@@ -0,0 +1,96 @@
|
||||
# Fleet Generated Environment Boundary
|
||||
|
||||
**Card:** FCM-M2-001 · **Issue:** #758 · **Status:** merged contract
|
||||
|
||||
The local fleet roster is the desired-state authority. A launch reads a deterministic,
|
||||
roster-derived generated projection and an optional strictly data-only local file; neither file is
|
||||
a second roster or a command configuration surface.
|
||||
|
||||
## Paths and ownership
|
||||
|
||||
For agent <name> under <MOSAIC_HOME>/fleet/agents/:
|
||||
|
||||
| Path | Owner | Purpose |
|
||||
| --------------------- | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------- |
|
||||
| <name>.env.generated | Mosaic projection writer | Complete deterministic launch data rendered from the authoritative roster. |
|
||||
| <name>.env.local | Operator | Optional, constrained local machine data. It cannot shadow generated keys. |
|
||||
| <name>.env | Legacy input only | Read once during projection generation, then regenerated/relocated or privately quarantined. It is never a launch authority. |
|
||||
| <name>.env.quarantine | Mosaic quarantine | Mode-`0600` private record of forbidden legacy input; it is never read by the launcher. |
|
||||
|
||||
The systemd templates do not load either environment file. They invoke Bash with a fixed, cleared
|
||||
bootstrap environment; the launcher reads and validates `.env.generated` and `.env.local` itself before
|
||||
it queries, creates, or stops an exact tmux session. It does not `source`, `eval`, or execute an
|
||||
environment-supplied command. Exact stop derives its socket from the same validated generated projection,
|
||||
not from systemd or ambient environment data.
|
||||
|
||||
All projection, local, and quarantine files must be regular files with no group or world permissions.
|
||||
The agent environment directory must also be a real, non-symlink private directory; it is validated
|
||||
before either environment file is read or tmux is queried. Unsafe paths, symlinks, or permissions fail
|
||||
closed. Diagnostics identify only a rule code, key name, and SHA-256 content hash; they never print
|
||||
values, credential material, or command text.
|
||||
|
||||
## Allowed data
|
||||
|
||||
`.env.generated` is complete and ordered exactly as follows:
|
||||
|
||||
```dotenv
|
||||
MOSAIC_AGENT_NAME=<roster name>
|
||||
MOSAIC_AGENT_CLASS=<roster class>
|
||||
MOSAIC_AGENT_RUNTIME=<roster runtime>
|
||||
MOSAIC_AGENT_MODEL=<roster model hint>
|
||||
MOSAIC_AGENT_REASONING=<roster reasoning>
|
||||
MOSAIC_AGENT_TOOL_POLICY=<roster tool policy>
|
||||
MOSAIC_AGENT_WORKDIR=<absolute roster work directory>
|
||||
MOSAIC_TMUX_SOCKET=<roster socket or empty>
|
||||
```
|
||||
|
||||
The generated launch contract supports only `claude`, `codex`, `opencode`, and `pi`. fleet add
|
||||
uses that same runtime authority and rejects any other runtime before it writes the roster or changes
|
||||
projection, local, or quarantine files. The legacy dogfood stub on its separate `mosaic-factory`
|
||||
socket remains an observability canary; it has no generated-launch adapter and cannot be added through
|
||||
this projection path.
|
||||
|
||||
`.env.local` may contain only these non-secret data keys:
|
||||
|
||||
- `MOSAIC_RUNTIME_BIN`
|
||||
- `MOSAIC_HEARTBEAT_RUN_DIR`
|
||||
- `MOSAIC_HEARTBEAT_INTERVAL`
|
||||
- `MOSAIC_CLAUDE_JSON`
|
||||
- `CLAUDE_CONFIG_DIR`
|
||||
|
||||
Local paths must be safe absolute paths and the interval must be a positive integer. Comments,
|
||||
quoted/export syntax, duplicate keys, unknown keys, generated-key shadowing, sensitive key names,
|
||||
and `MOSAIC_AGENT_COMMAND` are rejected. The launcher derives the only executable command from the
|
||||
validated runtime, model, and reasoning data; no arbitrary command compatibility path exists. When a
|
||||
Pi runtime writes a fresh <name>.hb.native marker, its native heartbeat remains authoritative; the
|
||||
shell sidecar resumes its `status=ok` fallback only after that marker is stale or absent.
|
||||
|
||||
## Legacy disposition
|
||||
|
||||
During projection generation, legacy roster-derived keys are regenerated from the roster. A valid
|
||||
allowed local value is relocated to `.env.local`; forbidden, malformed, duplicate, sensitive, and
|
||||
unknown legacy entries cause the legacy file to be moved to `.env.quarantine` and are represented by
|
||||
sanitized diagnostics. This is deterministic and idempotent after the legacy file has been consumed.
|
||||
|
||||
## USC interface packet
|
||||
|
||||
This card does not add a USC site file, write a USC roster, or run a site canary. The following is the
|
||||
consolidated downstream interface packet. Status is deliberately separated from checkout presence: no
|
||||
product release version has been evidenced for this interface set.
|
||||
|
||||
| Interface | Canonical public path and version | Tracker/release status | Downstream limit |
|
||||
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
|
||||
| M1 structural compiler | `parseRosterV2` in `packages/mosaic/src/fleet/roster-v2.ts`; schema `docs/fleet/reference/roster-v2.schema.json`; roster version: 2 | FCM-M1-001 is recorded done, merged as #764 (`aa5b43b`); no released product version is asserted here. | Parse YAML/JSON and canonicalize a supplied v2 site roster without writes. |
|
||||
| M1 semantic resolver | `validateRosterV2Semantics` in `packages/mosaic/src/fleet/roster-v2.ts`; baseline `framework/fleet/roles/` plus `roles.local/` | FCM-M1-002 merged as #768 (`a5e8e55`); no released product version is asserted here. | Reuse the shared resolver only; no parallel role resolver or lifecycle action. |
|
||||
| M1 disposition evidence | `packages/mosaic/src/fleet/example-profile-dispositions.ts`; `docs/fleet/migration/example-profile-disposition.md`; retained fixture version: 1 | FCM-M1-003 merged as #770 (`e9c4aa3`); checkout evidence remains validation, not migration authorization. | Inspect fixture/profile/service disposition evidence only; it is not migration authorization. |
|
||||
| M2 generated boundary | `packages/mosaic/src/fleet/generated-env-boundary.ts`; generated projection contract in this document | FCM-M2-001 merged as #772 (`191efae`); no released product version is asserted here. | Render/write a roster-derived projection; local input is never authority. |
|
||||
|
||||
The canonical source remains <MOSAIC_HOME>/fleet/roster.yaml for the current local fleet path.
|
||||
Generated environment data is a rebuildable projection, not an operator-editable source of membership,
|
||||
runtime policy, or lifecycle state.
|
||||
|
||||
**Corrected downstream gates:** M2 supplies only parse/validation/projection evidence and does not
|
||||
permit a USC site canary, reconciliation, or lifecycle mutation. M3 must first define and validate the
|
||||
canonical local reconcile/lifecycle path. M4 then supplies preview/migration and its separate
|
||||
canary/rollback gates; only after those M3 and M4 gates may a site migration or canary be considered.
|
||||
This card authorizes none of those actions.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Local Fleet Lifecycle Transitions
|
||||
|
||||
Roster-v2 `lifecycle.enabled` and `lifecycle.desired_state` are the only persisted lifecycle authority. Systemd, tmux, generated environment, and heartbeat state are derived or observed.
|
||||
|
||||
| Event | Desired-state write | Runtime effect | Safety boundary |
|
||||
| ------------------------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| fleet create | Adds enabled/stopped by default; `--persisted-start` records running | None | Generation-guarded; validates full roster/projections. |
|
||||
| fleet update | Preserves the existing enabled/desired state; updates other mutable fields | None | Generation-guarded; stable name and lifecycle are immutable on this path. |
|
||||
| fleet delete | Removes exact roster member | None | Removes only generated projection; retains local/quarantine evidence. |
|
||||
| fleet apply / `reconcile` | Never | Rebuilds projections; starts only enabled/running; stops disabled or stopped roster members | Current generation, private lock/paths, semantic validity, holder ownership, no unmanaged named-socket sessions. |
|
||||
| fleet start <name> | Never | One-shot exact service start | Exact enabled roster name and proven ownership. |
|
||||
| fleet stop <name> | Never | One-shot exact service stop | Exact roster name and proven ownership. |
|
||||
| fleet restart <name> | Never | One-shot exact service restart | Exact enabled roster name and proven ownership. |
|
||||
| Reboot/service activation | Never | Current installation may activate enabled units without honoring roster lifecycle | **Held for FCM-M3-002:** boot preservation for stopped/disabled agents is not yet proven; inspect/disable units rather than assuming lifecycle-safe reboot. |
|
||||
| v1 migration preview | Never | None | Observed active+present maps running; inactive+missing maps stopped; ambiguity blocks. |
|
||||
| Cutover/canary | Held for FCM-M4-002 | Not implemented by preview | Must preserve every observed stopped state. |
|
||||
| Rollback | Held for FCM-M4-002 | Not implemented | Must restore selected authority/projections without surprise starts or unmanaged targeting. |
|
||||
|
||||
Explicit apply/reconcile never starts a stopped roster agent. Direct lifecycle commands are explicit one-shot actions and do not persist intent. The current update operation preserves `existing.lifecycle`; there is no delivered generation-guarded CRUD operation for changing durable lifecycle after creation. Reboot preservation for stopped/disabled agents is not yet guaranteed because current enabled units and launcher projections do not carry the persisted lifecycle fence; that acceptance evidence remains FCM-M3-002.
|
||||
|
||||
Missing/stale generation, concurrent writer, unsafe path, ownership mismatch, unmanaged session, unsupported runtime, invalid projection, and lifecycle precondition failures return stable redacted JSON and non-zero status. No command targets fuzzy names, arbitrary sockets/commands/channels/secrets, or generated files as authority. Legacy sensitive values are never printed.
|
||||
@@ -0,0 +1,45 @@
|
||||
# Fleet Role Classes and Authority
|
||||
|
||||
A fleet role class is a machine identity resolved from the persona library. Resolution uses the
|
||||
canonical class before consulting the baseline `fleet/roles/` and operator `fleet/roles.local/`
|
||||
layers. A readable role contract is required; an index entry alone is not semantic success.
|
||||
|
||||
## Canonicalization
|
||||
|
||||
Only these legacy class aliases are recognized:
|
||||
|
||||
| Requested class | Canonical class |
|
||||
| ---------------------- | --------------- |
|
||||
| `implementer` | `code` |
|
||||
| `reviewer` | `review` |
|
||||
| `operator-interaction` | `interaction` |
|
||||
|
||||
No other alias is inferred. In particular, `worker`, `analyst`, and `canary` are custom classes only
|
||||
when an operator supplies a readable contract for that exact class. Tess and Ultron are instance
|
||||
names, not classes. agents[].alias is display-only and cannot grant authority.
|
||||
|
||||
Canonicalization happens before role lookup. For example, requesting `implementer` resolves
|
||||
`code.md`; a separate `roles.local/implementer.md` cannot redefine the legacy alias. A canonical
|
||||
`roles.local/code.md` still overrides the baseline `roles/code.md` contract.
|
||||
|
||||
## Protected authority
|
||||
|
||||
Protected authority is immutable metadata derived only from canonical class. Role prose, instance
|
||||
name, display alias, tool policy, runtime, and custom role files cannot grant it.
|
||||
|
||||
| Canonical class | Granted authority | Explicit limits |
|
||||
| ----------------- | -------------------------------------------------- | --------------------------------------------------------------------------------- |
|
||||
| `merge-gate` | Sole approve-to-land and merge authority | No authority is inferred by similarly named custom roles or policies. |
|
||||
| `validator` | May issue a validation certificate | Cannot approve-to-land or merge. |
|
||||
| `orchestrator` | May orchestrate, manage topology, and issue leases | Cannot approve-to-land or merge. |
|
||||
| `team-leader` | May use orchestrator-leased capacity | Cannot issue leases or mutate roster, configuration, credentials, or merge state. |
|
||||
| `interaction` | Request and status surface | Cannot orchestrate, issue leases, mutate roster/configuration, or merge. |
|
||||
| all other classes | No protected authority implicitly | Custom contracts do not acquire protected powers from prose. |
|
||||
|
||||
Roster-v2 semantic validation requires a protected class and its canonical tool policy to match. It
|
||||
also rejects an unprotected class paired with a protected tool policy. The legacy tool-policy name
|
||||
`operator-interaction` canonicalizes to `interaction`.
|
||||
|
||||
This mapping describes authority metadata only. Lease issuance, validation-certificate storage or
|
||||
workflow, lifecycle reconciliation, credentials, roster mutation, and merge execution are outside
|
||||
this resolver contract.
|
||||
@@ -0,0 +1,126 @@
|
||||
# Fleet Roster v2 Structural Contract
|
||||
|
||||
**Status:** FCM-M1-001 local-tmux structural compiler contract. This document describes parsing,
|
||||
strict structural validation, normalized in-memory representation, and deterministic rendering only.
|
||||
It does not authorize role resolution, lifecycle reconciliation, mutation, migration, remote
|
||||
placement, connector configuration, secret references, arbitrary commands, channels, gateway
|
||||
mapping, or any live-fleet change.
|
||||
|
||||
The executable schema is [`roster-v2.schema.json`](./roster-v2.schema.json). The compiler exports
|
||||
the same schema and its test parses this file and compares it structurally with the executable contract.
|
||||
|
||||
## Format and canonical shape
|
||||
|
||||
The compiler accepts YAML or JSON. It reads only snake_case source fields and renders canonical,
|
||||
snake_case YAML. Rendering sorts runtime keys and agents by stable name. Agent names, class names,
|
||||
and tool-policy names are structural identifiers; whether a class or policy resolves is a later
|
||||
shared-resolver concern.
|
||||
|
||||
```yaml
|
||||
version: 2
|
||||
generation: 1
|
||||
transport: tmux
|
||||
tmux:
|
||||
socket_name: mosaic-fleet
|
||||
holder_session: _holder
|
||||
defaults:
|
||||
working_directory: ~/src
|
||||
runtime: pi
|
||||
runtimes:
|
||||
pi:
|
||||
reset_command: /new
|
||||
agents:
|
||||
- name: coder0
|
||||
alias: Coder 0
|
||||
class: code
|
||||
runtime: pi
|
||||
provider: openai
|
||||
model: gpt-5.6-sol
|
||||
reasoning: high
|
||||
tool_policy: code
|
||||
working_directory: ~/src
|
||||
persistent_persona: false
|
||||
reset_between_tasks: true
|
||||
lifecycle:
|
||||
enabled: true
|
||||
desired_state: stopped
|
||||
launch:
|
||||
yolo: true
|
||||
```
|
||||
|
||||
## Root fields
|
||||
|
||||
| Field | Required | Default | Constraint | Meaning |
|
||||
| ------------ | -------- | ------- | --------------------- | ------------------------------------------------------------------------------------------------- |
|
||||
| `version` | yes | none | integer constant `2` | Identifies this contract. Version `1` stays on the compatibility path pending explicit migration. |
|
||||
| `generation` | yes | none | positive safe integer | Desired-state generation and mutation/reconcile concurrency fence. |
|
||||
| `transport` | yes | none | constant `tmux` | M1–M5 support local tmux only. |
|
||||
| `tmux` | yes | none | strict object | Explicit local socket and holder-session configuration. |
|
||||
| `defaults` | yes | none | strict object | Default work directory and one supported local runtime. |
|
||||
| `runtimes` | yes | none | non-empty object | Declared local runtime reset policy map. |
|
||||
| `agents` | yes | none | non-empty array | Local fleet entries. Duplicate stable names are rejected. |
|
||||
|
||||
## Nested fields
|
||||
|
||||
All nested fields in the v2 schema are required and have no implicit default. CRUD `create` is the only higher-level convenience: it records lifecycle.enabled: true and desired_state: stopped unless `--persisted-start` explicitly records running. That convenience still performs no runtime action.
|
||||
|
||||
| Path | Required | Constraint |
|
||||
| -------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
| `tmux.socket_name` | yes | [A-Za-z0-9_.-]\*; empty string means the literal default tmux server, while a non-empty value names a socket |
|
||||
| `tmux.holder_session` | yes | non-empty [A-Za-z0-9_.-]+ |
|
||||
| `defaults.working_directory` | yes | non-empty string |
|
||||
| `defaults.runtime` | yes | `claude`, `codex`, `opencode`, or `pi`; it must be declared in `runtimes` |
|
||||
| runtimes.<runtime>.reset_command | yes | non-empty string; runtime key must be a supported local runtime |
|
||||
| agents[].name | yes | unique [A-Za-z0-9][A-Za-z0-9_.-]\* stable machine identity |
|
||||
| agents[].alias | yes | non-empty display string |
|
||||
| agents[].class | yes | [a-z][a-z0-9-]\*; structural only in M1, semantic role resolution is FCM-M1-002 |
|
||||
| agents[].runtime | yes | `claude`, `codex`, `opencode`, or `pi`; it must be declared in `runtimes` |
|
||||
| agents[].provider, `model`, `working_directory` | yes | non-empty strings; provider/model capability resolution is a later card |
|
||||
| agents[].reasoning | yes | `low`, `medium`, or `high` |
|
||||
| agents[].tool_policy | yes | [a-z][a-z0-9-]\*; structural only in M1 |
|
||||
| agents[].persistent_persona, `reset_between_tasks` | yes | booleans |
|
||||
| agents[].lifecycle.enabled | yes | boolean; stored now, reconciled in FCM-M3-001 |
|
||||
| agents[].lifecycle.desired_state | yes | `running` or `stopped` |
|
||||
| agents[].launch.yolo | yes | boolean; structured data only, not an arbitrary command escape hatch |
|
||||
|
||||
## Semantic handoff
|
||||
|
||||
`parseRosterV2` and `normalizeRosterV2` remain synchronous and structural. After structural success,
|
||||
call the asynchronous `validateRosterV2Semantics` handoff before using persona identity or authority.
|
||||
That validator batches the baseline `fleet/roles/` and operator `fleet/roles.local/` scans, then
|
||||
delegates every agent to the shared persona resolver.
|
||||
|
||||
Semantic validation:
|
||||
|
||||
- requires the winning role contract to be readable and non-empty; `LIBRARY.md` membership alone does
|
||||
not resolve a class;
|
||||
- retains `requestedClass` separately from `canonicalClass` in typed output;
|
||||
- canonicalizes only `implementer` to `code`, `reviewer` to `review`, and
|
||||
`operator-interaction` to `interaction`;
|
||||
- canonicalizes `tool_policy` with the same exact alias table;
|
||||
- rejects protected class/tool-policy mismatches in either direction, while accepting
|
||||
class: operator-interaction with tool_policy: operator-interaction as canonical
|
||||
`interaction`;
|
||||
- derives immutable protected authority only from canonical class; and
|
||||
- accepts custom baseline or `roles.local` classes without granting protected authority.
|
||||
|
||||
agents[].alias remains display-only. Tess and Ultron are instance names, never semantic classes.
|
||||
Canonicalization happens before role-layer lookup, so a legacy-named override cannot redefine an
|
||||
alias as separate authority. See [Role Classes and Authority](./role-classes.md) and
|
||||
[Customize Fleet Roles](../how-to/customize-roles.md).
|
||||
|
||||
This handoff performs no filesystem, systemd, tmux, roster, credential, lease, certificate, or
|
||||
lifecycle mutation.
|
||||
|
||||
## Fail-closed boundary
|
||||
|
||||
Every object is additionalProperties: false. The compiler rejects unknown, missing, malformed,
|
||||
and wrong-type fields before producing a model. It specifically rejects remote/SSH/host/socket
|
||||
per-agent fields, connector blocks, secret references, channel fields, arbitrary command fields,
|
||||
and gateway fields because they are unsupported in the local-tmux M1 contract. It does not silently
|
||||
ignore v1 camelCase input, version `1`, or a source that does not parse to an object.
|
||||
|
||||
The v2 compiler is intentionally isolated from the existing v1 loader. Existing v1 rosters and
|
||||
current examples/profiles continue on their current path; FCM-M4 owns explicit inventory, preview,
|
||||
migration, and rollback. FCM-M2 owns generated-file/local-override quarantine, and FCM-M3 owns
|
||||
runtime lifecycle and reconciliation.
|
||||
@@ -0,0 +1,156 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://mosaicstack.dev/schemas/fleet/roster-v2.schema.json",
|
||||
"title": "Mosaic local tmux fleet roster v2",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["version", "generation", "transport", "tmux", "defaults", "runtimes", "agents"],
|
||||
"properties": {
|
||||
"version": {
|
||||
"const": 2
|
||||
},
|
||||
"generation": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 9007199254740991
|
||||
},
|
||||
"transport": {
|
||||
"const": "tmux"
|
||||
},
|
||||
"tmux": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["socket_name", "holder_session"],
|
||||
"properties": {
|
||||
"socket_name": {
|
||||
"type": "string",
|
||||
"pattern": "^[A-Za-z0-9_.-]*$"
|
||||
},
|
||||
"holder_session": {
|
||||
"type": "string",
|
||||
"pattern": "^[A-Za-z0-9_.-]+$"
|
||||
}
|
||||
}
|
||||
},
|
||||
"defaults": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["working_directory", "runtime"],
|
||||
"properties": {
|
||||
"working_directory": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"runtime": {
|
||||
"enum": ["claude", "codex", "opencode", "pi"]
|
||||
}
|
||||
}
|
||||
},
|
||||
"runtimes": {
|
||||
"type": "object",
|
||||
"minProperties": 1,
|
||||
"propertyNames": {
|
||||
"enum": ["claude", "codex", "opencode", "pi"]
|
||||
},
|
||||
"additionalProperties": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["reset_command"],
|
||||
"properties": {
|
||||
"reset_command": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"agents": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"name",
|
||||
"alias",
|
||||
"class",
|
||||
"runtime",
|
||||
"provider",
|
||||
"model",
|
||||
"reasoning",
|
||||
"tool_policy",
|
||||
"working_directory",
|
||||
"persistent_persona",
|
||||
"reset_between_tasks",
|
||||
"lifecycle",
|
||||
"launch"
|
||||
],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"pattern": "^[A-Za-z0-9][A-Za-z0-9_.-]*$"
|
||||
},
|
||||
"alias": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"class": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z][a-z0-9-]*$"
|
||||
},
|
||||
"runtime": {
|
||||
"enum": ["claude", "codex", "opencode", "pi"]
|
||||
},
|
||||
"provider": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"reasoning": {
|
||||
"enum": ["low", "medium", "high"]
|
||||
},
|
||||
"tool_policy": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z][a-z0-9-]*$"
|
||||
},
|
||||
"working_directory": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"persistent_persona": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"reset_between_tasks": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"lifecycle": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["enabled", "desired_state"],
|
||||
"properties": {
|
||||
"enabled": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"desired_state": {
|
||||
"enum": ["running", "stopped"]
|
||||
}
|
||||
}
|
||||
},
|
||||
"launch": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["yolo"],
|
||||
"properties": {
|
||||
"yolo": {
|
||||
"type": "boolean"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
# Local Fleet Status and Drift
|
||||
|
||||
mosaic fleet status [<name>], `verify`, and `doctor` are observational roster-v2 commands. They emit one JSON result and do not write projections, mutate desired state, operate lifecycle, or change tmux.
|
||||
|
||||
## State dimensions
|
||||
|
||||
- **Desired:** roster membership, generation, enabled flag, and persisted running/stopped target.
|
||||
- **Managed/derived:** generated environment and expected exact service/session topology.
|
||||
- **Observed by current roster-v2 commands:** systemd active state, tmux presence, exact holder ownership, and unmanaged sessions.
|
||||
|
||||
Implemented drift classifications include:
|
||||
|
||||
- `missing-session`: enabled/desired-running agent lacks its exact session;
|
||||
- `unexpected-session`: desired-stopped agent has its exact session;
|
||||
- `disabled-running`: disabled roster agent has its exact session;
|
||||
- `unmanagedSessions`: named-socket sessions that are neither exact holder nor roster agent;
|
||||
- `holder`: `owned`, `missing`, or `ownership-mismatch` after private identity and global environment checks.
|
||||
|
||||
Generated projection failures/staleness are surfaced by plan/apply preparation and bounded recovery fields rather than adopted as configuration. Heartbeat remains wider-fleet observational evidence, never desired state, but the current roster-v2 `status`, `doctor`, and `verify` commands do not read heartbeat files. A provable removed-agent projection may be treated as stale derived state during deletion, but general projection-orphan classification and installed source-versus-asset revision mismatch remain FCM-M4-002/M5-002 holds; current commands must not claim those future checks.
|
||||
|
||||
## Command behavior
|
||||
|
||||
`status` and `doctor` classify rather than adopt, destroy, or repair. `verify` is observational too, but exits non-zero if ownership cannot be proven, unmanaged sessions exist, or drift is present. Reconciliation fails closed under those conditions and never kills or adopts an unmanaged session.
|
||||
|
||||
Doctor/error output uses stable codes and bounded recovery context. Migration, quarantine, lifecycle, status, and troubleshooting output never prints a legacy sensitive value, credential, or privileged command text.
|
||||
@@ -0,0 +1,376 @@
|
||||
# Mosaic Stack — Admin Guide
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [User Management](#user-management)
|
||||
2. [System Health Monitoring](#system-health-monitoring)
|
||||
3. [Provider Configuration](#provider-configuration)
|
||||
4. [MCP Server Configuration](#mcp-server-configuration)
|
||||
5. [Environment Variables Reference](#environment-variables-reference)
|
||||
6. [Local Fleet Canary](./fleet-local-canary.md)
|
||||
|
||||
---
|
||||
|
||||
## User Management
|
||||
|
||||
Admins access user management at `/admin` in the web dashboard. All admin
|
||||
endpoints require a session with `role = admin`.
|
||||
|
||||
### Creating a User
|
||||
|
||||
**Via the web admin panel:**
|
||||
|
||||
1. Navigate to `/admin`.
|
||||
2. Click **Create User**.
|
||||
3. Enter name, email, password, and role (`admin` or `member`).
|
||||
4. Submit.
|
||||
|
||||
**Via the API:**
|
||||
|
||||
```http
|
||||
POST /api/admin/users
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"name": "Jane Doe",
|
||||
"email": "jane@example.com",
|
||||
"password": "securepassword",
|
||||
"role": "member"
|
||||
}
|
||||
```
|
||||
|
||||
Passwords are hashed by BetterAuth before storage. Passwords are never stored in
|
||||
plaintext.
|
||||
|
||||
### Roles
|
||||
|
||||
| Role | Permissions |
|
||||
| -------- | --------------------------------------------------------------------- |
|
||||
| `admin` | Full access: user management, health, all agent tools |
|
||||
| `member` | Standard user access; agent tool set restricted by `AGENT_USER_TOOLS` |
|
||||
|
||||
### Updating a User's Role
|
||||
|
||||
```http
|
||||
PATCH /api/admin/users/:id/role
|
||||
Content-Type: application/json
|
||||
|
||||
{ "role": "admin" }
|
||||
```
|
||||
|
||||
### Banning and Unbanning
|
||||
|
||||
Banned users cannot sign in. Provide an optional reason:
|
||||
|
||||
```http
|
||||
POST /api/admin/users/:id/ban
|
||||
Content-Type: application/json
|
||||
|
||||
{ "reason": "Violated terms of service" }
|
||||
```
|
||||
|
||||
To lift a ban:
|
||||
|
||||
```http
|
||||
POST /api/admin/users/:id/unban
|
||||
```
|
||||
|
||||
### Deleting a User
|
||||
|
||||
```http
|
||||
DELETE /api/admin/users/:id
|
||||
```
|
||||
|
||||
This permanently deletes the user. Related data (sessions, accounts) is
|
||||
cascade-deleted. Conversations and tasks reference the user via `owner_id`
|
||||
which is set to `NULL` on delete (`set null`).
|
||||
|
||||
---
|
||||
|
||||
## System Health Monitoring
|
||||
|
||||
The health endpoint is available to admin users only.
|
||||
|
||||
```http
|
||||
GET /api/admin/health
|
||||
```
|
||||
|
||||
Sample response:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"database": { "status": "ok", "latencyMs": 2 },
|
||||
"cache": { "status": "ok", "latencyMs": 1 },
|
||||
"agentPool": { "activeSessions": 3 },
|
||||
"providers": [{ "id": "ollama", "name": "ollama", "available": true, "modelCount": 3 }],
|
||||
"checkedAt": "2026-03-15T12:00:00.000Z"
|
||||
}
|
||||
```
|
||||
|
||||
`status` is `ok` when both database and cache pass. It is `degraded` when either
|
||||
service fails.
|
||||
|
||||
The web admin panel at `/admin` polls this endpoint and renders the results in a
|
||||
status dashboard.
|
||||
|
||||
---
|
||||
|
||||
## Provider Configuration
|
||||
|
||||
Providers are configured via environment variables and loaded at gateway startup.
|
||||
No restart-free hot reload is supported; the gateway must be restarted after
|
||||
changing provider env vars.
|
||||
|
||||
### Ollama
|
||||
|
||||
Set `OLLAMA_BASE_URL` (or the legacy `OLLAMA_HOST`) to the base URL of your
|
||||
Ollama instance:
|
||||
|
||||
```env
|
||||
OLLAMA_BASE_URL=http://localhost:11434
|
||||
```
|
||||
|
||||
Specify which models to expose (comma-separated):
|
||||
|
||||
```env
|
||||
OLLAMA_MODELS=llama3.2,codellama,mistral
|
||||
```
|
||||
|
||||
Default when unset: `llama3.2,codellama,mistral`.
|
||||
|
||||
The gateway registers Ollama models using the OpenAI-compatible completions API
|
||||
(`/v1/chat/completions`).
|
||||
|
||||
### Custom Providers (OpenAI-compatible APIs)
|
||||
|
||||
Any OpenAI-compatible API (LM Studio, llama.cpp HTTP server, etc.) can be
|
||||
registered via `MOSAIC_CUSTOM_PROVIDERS`. The value is a JSON array:
|
||||
|
||||
```env
|
||||
MOSAIC_CUSTOM_PROVIDERS='[
|
||||
{
|
||||
"id": "lmstudio",
|
||||
"name": "LM Studio",
|
||||
"baseUrl": "http://localhost:1234",
|
||||
"models": ["mistral-7b-instruct"]
|
||||
}
|
||||
]'
|
||||
```
|
||||
|
||||
Each entry must include:
|
||||
|
||||
| Field | Required | Description |
|
||||
| --------- | -------- | ----------------------------------- |
|
||||
| `id` | Yes | Unique provider identifier |
|
||||
| `name` | Yes | Display name |
|
||||
| `baseUrl` | Yes | API base URL (no trailing slash) |
|
||||
| `models` | Yes | Array of model ID strings to expose |
|
||||
| `apiKey` | No | API key if required by the endpoint |
|
||||
|
||||
### Testing Provider Connectivity
|
||||
|
||||
From the web admin panel or settings page, click **Test** next to a provider.
|
||||
This calls:
|
||||
|
||||
```http
|
||||
POST /api/agent/providers/:id/test
|
||||
```
|
||||
|
||||
The response includes `reachable`, `latencyMs`, and optionally
|
||||
`discoveredModels`.
|
||||
|
||||
---
|
||||
|
||||
## MCP Server Configuration
|
||||
|
||||
The gateway can connect to external MCP (Model Context Protocol) servers and
|
||||
expose their tools to agent sessions.
|
||||
|
||||
Set `MCP_SERVERS` to a JSON array of server configurations:
|
||||
|
||||
```env
|
||||
MCP_SERVERS='[
|
||||
{
|
||||
"name": "my-tools",
|
||||
"url": "http://localhost:3001/mcp",
|
||||
"headers": {
|
||||
"Authorization": "Bearer my-token"
|
||||
}
|
||||
}
|
||||
]'
|
||||
```
|
||||
|
||||
Each entry:
|
||||
|
||||
| Field | Required | Description |
|
||||
| --------- | -------- | ----------------------------------- |
|
||||
| `name` | Yes | Unique server name |
|
||||
| `url` | Yes | MCP server URL (`/mcp` endpoint) |
|
||||
| `headers` | No | Additional HTTP headers (e.g. auth) |
|
||||
|
||||
On gateway startup, each configured server is connected and its tools are
|
||||
discovered. Tools are bridged into the Pi SDK tool format and become available
|
||||
in agent sessions.
|
||||
|
||||
The gateway itself also exposes an MCP server endpoint at `POST /mcp` for
|
||||
external clients. Authentication requires a valid BetterAuth session (cookie or
|
||||
`Authorization` header).
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables Reference
|
||||
|
||||
### Required
|
||||
|
||||
| Variable | Description |
|
||||
| -------------------- | ----------------------------------------------------------------------------------------------------------- |
|
||||
| `BETTER_AUTH_SECRET` | Secret key for BetterAuth session signing. Must be set or gateway will not start. |
|
||||
| `DATABASE_URL` | Runtime-only PostgreSQL connection injected from the dedicated deployment secret; no default or inline DSN. |
|
||||
|
||||
### Gateway
|
||||
|
||||
| Variable | Default | Description |
|
||||
| --------------------- | ------------------------ | ---------------------------------------------- |
|
||||
| `GATEWAY_PORT` | `14242` | Port the gateway listens on |
|
||||
| `GATEWAY_CORS_ORIGIN` | `http://localhost:3000` | Allowed CORS origin for browser clients |
|
||||
| `BETTER_AUTH_URL` | `http://localhost:14242` | Public URL of the gateway (used by BetterAuth) |
|
||||
|
||||
### SSO (Optional)
|
||||
|
||||
| Variable | Description |
|
||||
| --------------------------- | ---------------------------------------------------------------------------- |
|
||||
| `AUTHENTIK_CLIENT_ID` | Authentik OAuth2 client ID |
|
||||
| `AUTHENTIK_CLIENT_SECRET` | Authentik OAuth2 client secret |
|
||||
| `AUTHENTIK_ISSUER` | Authentik OIDC issuer URL |
|
||||
| `AUTHENTIK_TEAM_SYNC_CLAIM` | Optional claim used to derive team sync data (defaults to `groups`) |
|
||||
| `WORKOS_CLIENT_ID` | WorkOS OAuth client ID |
|
||||
| `WORKOS_CLIENT_SECRET` | WorkOS OAuth client secret |
|
||||
| `WORKOS_ISSUER` | WorkOS OIDC issuer URL |
|
||||
| `WORKOS_TEAM_SYNC_CLAIM` | Optional claim used to derive team sync data (defaults to `organization_id`) |
|
||||
| `KEYCLOAK_CLIENT_ID` | Keycloak OAuth client ID |
|
||||
| `KEYCLOAK_CLIENT_SECRET` | Keycloak OAuth client secret |
|
||||
| `KEYCLOAK_ISSUER` | Keycloak realm issuer URL |
|
||||
| `KEYCLOAK_TEAM_SYNC_CLAIM` | Optional claim used to derive team sync data (defaults to `groups`) |
|
||||
| `KEYCLOAK_SAML_LOGIN_URL` | Optional SAML login URL used when OIDC is unavailable |
|
||||
|
||||
Each OIDC provider requires its client ID, client secret, and issuer URL together. If only part of a provider configuration is set, gateway startup logs a warning and that provider is skipped. Keycloak can fall back to SAML when `KEYCLOAK_SAML_LOGIN_URL` is configured.
|
||||
|
||||
### Agent
|
||||
|
||||
| Variable | Default | Description |
|
||||
| ------------------------ | --------------- | ------------------------------------------------------- |
|
||||
| `AGENT_FILE_SANDBOX_DIR` | `process.cwd()` | Root directory for file/git/shell tool access |
|
||||
| `AGENT_SYSTEM_PROMPT` | — | Platform-level system prompt injected into all sessions |
|
||||
| `AGENT_USER_TOOLS` | all tools | Comma-separated allowlist of tools for non-admin users |
|
||||
|
||||
### Providers
|
||||
|
||||
| Variable | Default | Description |
|
||||
| ------------------------- | ---------------------------- | ------------------------------------------------ |
|
||||
| `OLLAMA_BASE_URL` | — | Ollama API base URL |
|
||||
| `OLLAMA_HOST` | — | Alias for `OLLAMA_BASE_URL` (legacy) |
|
||||
| `OLLAMA_MODELS` | `llama3.2,codellama,mistral` | Comma-separated Ollama model IDs |
|
||||
| `MOSAIC_CUSTOM_PROVIDERS` | — | JSON array of custom OpenAI-compatible providers |
|
||||
|
||||
### Memory and Embeddings
|
||||
|
||||
| Variable | Default | Description |
|
||||
| ----------------------- | --------------------------- | ---------------------------------------------------- |
|
||||
| `OPENAI_API_KEY` | — | API key for OpenAI embedding and summarization calls |
|
||||
| `EMBEDDING_API_URL` | `https://api.openai.com/v1` | Base URL for embedding API |
|
||||
| `EMBEDDING_MODEL` | `text-embedding-3-small` | Embedding model ID |
|
||||
| `SUMMARIZATION_API_URL` | `https://api.openai.com/v1` | Base URL for log summarization API |
|
||||
| `SUMMARIZATION_MODEL` | `gpt-4o-mini` | Model used for log summarization |
|
||||
| `SUMMARIZATION_CRON` | `0 */6 * * *` | Cron schedule for log summarization (every 6 hours) |
|
||||
| `TIER_MANAGEMENT_CRON` | `0 3 * * *` | Cron schedule for log tier management (daily at 3am) |
|
||||
|
||||
### MCP
|
||||
|
||||
| Variable | Description |
|
||||
| ------------- | ------------------------------------------------ |
|
||||
| `MCP_SERVERS` | JSON array of external MCP server configurations |
|
||||
|
||||
### Plugins
|
||||
|
||||
| Variable | Description |
|
||||
| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `DISCORD_BOT_TOKEN` | Discord bot token (enables Discord plugin) |
|
||||
| `DISCORD_SERVICE_TOKEN` | Required high-entropy service credential used to authenticate and sign Discord ingress; inject through the approved secret mechanism only |
|
||||
| `DISCORD_SERVICE_USER_ID` | Required Mosaic service-principal user ID that owns persisted Discord conversations; the original Discord user ID remains audit metadata |
|
||||
| `DISCORD_GUILD_ID` | Discord guild/server ID |
|
||||
| `DISCORD_GATEWAY_URL` | Gateway URL for Discord plugin to call (default: `http://localhost:14242`) |
|
||||
| `DISCORD_ALLOWED_GUILD_IDS` | Required comma-separated Discord guild snowflake allowlist; default-deny |
|
||||
| `DISCORD_ALLOWED_CHANNEL_IDS` | Required comma-separated Discord channel snowflake allowlist; default-deny |
|
||||
| `DISCORD_ALLOWED_USER_IDS` | Required comma-separated Discord user snowflake allowlist; default-deny |
|
||||
| `DISCORD_INTERACTION_BINDINGS` | Required JSON bindings from guild/channel to logical agent and paired Discord users with `viewer`, `operator`, or `admin` roles |
|
||||
| `DISCORD_MESSAGE_RATE_LIMIT_PER_MINUTE` | Optional positive integer; authorized turns per guild/channel/user each minute (default: `30`) |
|
||||
| `DISCORD_THREAD_RATE_LIMIT_PER_MINUTE` | Optional positive integer; mention-thread routes per guild/channel/user each minute (default: `5`) |
|
||||
| `TELEGRAM_BOT_TOKEN` | Telegram bot token (enables Telegram plugin) |
|
||||
| `TELEGRAM_GATEWAY_URL` | Gateway URL for Telegram plugin to call |
|
||||
|
||||
### Discord ingress security
|
||||
|
||||
When `DISCORD_BOT_TOKEN` is configured, `DISCORD_SERVICE_TOKEN`, `DISCORD_SERVICE_USER_ID`, and all three Discord allowlists are required. Gateway startup fails rather than enabling a broad or unauthenticated remote-control surface. The service user ID identifies a provisioned Mosaic service principal for persistence; the original Discord user ID is retained in ingress audit metadata. The service token is a secret supplied by the approved runtime secret mechanism and is never committed or logged.
|
||||
|
||||
Inbound Discord messages must originate from an allowed guild and configured parent channel, come from an allowed and paired user whose role permits sending, and carry a signed envelope containing the native Discord message ID and a generated correlation ID. Attachment references are limited in count, metadata size, field length, and declared size; only query-free HTTPS URLs without credentials or fragments are accepted, so bearer or presigned URLs never reach persistence or an agent prompt. The gateway validates the service identity, envelope signature, allowlists, pairing, and role again before dispatching. Replayed Discord message IDs are rejected during the bounded ingress replay window. Durable inbox/idempotency retention is introduced with Tess durable state.
|
||||
|
||||
Configured channels are dedicated agent interaction surfaces. An authorized untagged message routes to the bound logical agent and the response returns in that channel. Mentioning the bot on a normal channel message creates a public Discord thread, or reuses the thread already attached to that same message; the response and later thread messages stay in that thread without repeated mentions. A normal channel's category is not an authorization parent—only a Discord thread inherits authorization from its configured parent channel. Runtime control commands such as `/approve` and `/stop <approval>` remain on the current channel/thread because they target that durable session rather than opening a new topic.
|
||||
|
||||
Authorization and per-user/channel rate limits are evaluated before thread creation, so an unlisted guild/channel/user, unpaired user, `viewer`, or rate-limited sender cannot create bot threads or dispatch gateway work. The bot needs Discord permissions to view/send in configured channels and create/send in public threads. If thread creation fails, the turn is not dispatched because the requested response destination cannot be honored.
|
||||
|
||||
Conversation handles use the configured logical agent plus Discord channel/thread identity. They do not contain a Claude, Codex, Pi, OpenCode, model, process, or runtime-provider identifier; changing the runtime behind the logical session therefore does not require reconnecting the Discord bot.
|
||||
|
||||
#### Interaction binding format
|
||||
|
||||
`DISCORD_INTERACTION_BINDINGS` must be a non-empty JSON array. Each item requires `instanceId`, trusted `agentConfigId`, `guildId`, `channelId`, and a non-empty `pairedUsers` object. `instanceId` is the configured logical-agent name; its trusted database `agentConfigId` must resolve to an agent configuration with exactly that name, preserving provider/model/prompt/tool selection per binding. The guild/channel must also appear in their corresponding allowlists. IDs below are placeholders:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"instanceId": "interaction-agent",
|
||||
"agentConfigId": "agent-config-id",
|
||||
"guildId": "guild-id",
|
||||
"channelId": "channel-id",
|
||||
"pairedUsers": {
|
||||
"discord-user-id": {
|
||||
"role": "operator",
|
||||
"mosaicUserId": "mosaic-user-id"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
| Pairing role | Send message | Create/continue thread | Approve | Stop |
|
||||
| ------------ | ------------ | ---------------------- | ------- | ---- |
|
||||
| `viewer` | No | No | No | No |
|
||||
| `operator` | Yes | Yes | No | No |
|
||||
| `admin` | Yes | Yes | Yes | Yes |
|
||||
|
||||
A legacy role-only value such as `"discord-user-id": "operator"` remains valid for non-privileged ingress. Approval and stop require the object form with a provisioned `mosaicUserId`; gateway policy checks that Mosaic identity and consumes one exact-action approval once. Do not make the Discord service principal an approving administrator.
|
||||
|
||||
After changing bindings or allowlists, restart the gateway/plugin through the normal service manager and verify both Discord and gateway connectivity. The adapter reports `connected` only when both links are ready, `degraded` when one is ready, and `disconnected` when neither is ready. Test one authorized untagged channel turn, one mention-created thread, one thread follow-up, and one unauthorized user denial without using production credential values in logs or evidence.
|
||||
|
||||
### Session retention and garbage collection
|
||||
|
||||
Session cleanup is scoped to one session identifier and only removes that session's Valkey keys and demotes that session's hot logs. Gateway startup and scheduled jobs do not perform global session cleanup; startup removes legacy repeatable `session-gc` schedules created by older deployments. The `/gc` command is intentionally disabled until a distinct global-retention job supplies explicit authorization and audit evidence. This prevents one tenant or session's cleanup from changing another's retained data.
|
||||
|
||||
### Observability
|
||||
|
||||
| Variable | Default | Description |
|
||||
| ----------------------------- | ----------------------- | -------------------------------- |
|
||||
| `OTEL_EXPORTER_OTLP_ENDPOINT` | `http://localhost:4318` | OpenTelemetry collector endpoint |
|
||||
| `OTEL_SERVICE_NAME` | `mosaic-gateway` | Service name in traces |
|
||||
|
||||
### Web App
|
||||
|
||||
| Variable | Default | Description |
|
||||
| ------------------------- | ------------------------ | -------------------------------------- |
|
||||
| `NEXT_PUBLIC_GATEWAY_URL` | `http://localhost:14242` | Gateway URL used by the Next.js client |
|
||||
|
||||
### Coordination
|
||||
|
||||
| Variable | Default | Description |
|
||||
| ----------------------- | ----------------------------- | ------------------------------------------ |
|
||||
| `MOSAIC_WORKSPACE_ROOT` | monorepo root (auto-detected) | Root path for mission workspace operations |
|
||||
@@ -0,0 +1,67 @@
|
||||
# Deployment Guide
|
||||
|
||||
> **Status: non-operative for PostgreSQL, federated, and bare-metal production.** The checked-in
|
||||
> Compose PostgreSQL service mounts legacy initialization SQL and the KBN-101 bootstrap, runner,
|
||||
> secret-renderer, and process-exec interfaces do not exist yet. This page does not authorize a
|
||||
> production deployment, database initialization, manual DDL, secret provisioning, or service
|
||||
> activation.
|
||||
|
||||
## Current safe local route
|
||||
|
||||
Use PGlite only for current in-process data-layer work; it requires no PostgreSQL. A Gateway/Web
|
||||
local process is held because its unguarded dotenv loader can inherit a daemon PostgreSQL DSN and
|
||||
reach runtime DDL. If a local queue service is useful, start only Valkey:
|
||||
|
||||
```bash
|
||||
docker compose up -d valkey
|
||||
```
|
||||
|
||||
This command intentionally does not start PostgreSQL. Do not run a broad Compose start, use its
|
||||
PostgreSQL initialization mount, infer that current Compose is a production/federated route, or
|
||||
start Gateway/Web until KBN-101-02 supplies fail-closed local-tier/DSN isolation.
|
||||
|
||||
## Held future procedure
|
||||
|
||||
PostgreSQL local, federated, Compose, and bare-metal production activation are held until these
|
||||
artifacts land and pass their independent gates:
|
||||
|
||||
1. **KBN-101-00** external privileged bootstrap artifact;
|
||||
2. **KBN-101-03** sole `mosaic-db-migrator` runner and verified-readiness artifact; and
|
||||
3. **KBN-101-05** Vault/secret-renderer-backed deployment and consumer-isolation artifact.
|
||||
|
||||
The required future order is external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness.
|
||||
|
||||
This is a held, non-operative future activation specification with no current command authority. Do not invoke the named
|
||||
runner, start PostgreSQL, or substitute a Compose/init/manual-SQL route until the owned artifacts
|
||||
are implemented and reviewed.
|
||||
|
||||
## Future production secret and unit boundary (schematic only)
|
||||
|
||||
No current bare-metal production unit or command is published. KBN-101-05 must supply a reviewed,
|
||||
generation-pinned Vault renderer and a process-exec or systemd `LoadCredential` interface before
|
||||
production units can exist. The interface must preserve these exact consumer boundaries:
|
||||
|
||||
| Consumer | May receive | Must never receive |
|
||||
| ----------------- | -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
|
||||
| Gateway/runtime | Its own runtime URL and DB client CA at process exec | Migrator URL, importer URL/version, attestation material, signing key, PostgreSQL private key |
|
||||
| One-shot migrator | Its own migration URL, DB client CA, and runner-only signing capability | Runtime URL, importer consumer copy, Gateway/private PostgreSQL keys |
|
||||
| Data importer | Its own immutable URL/version copies, importer CA, pinned public key, and sealed attestation | Runtime/migrator URLs, signing key, shared writable mount |
|
||||
| PostgreSQL | Its own server certificate/key and only its approved server material | Application, migrator, importer, or Gateway secrets |
|
||||
|
||||
A future unit specification is non-executable until KBN-101-05 supplies it. It must obtain
|
||||
credentials through the renderer’s Vault generation and process-exec/`LoadCredential` boundary;
|
||||
it must not place credentials in a production environment file, a monorepo auto-load path, a shell
|
||||
export, command arguments, logs, or a manual secret-activation lifecycle instruction. Rotation and
|
||||
process replacement semantics must be delivered by the reviewed renderer/interface with generation,
|
||||
consumer-isolation, mode/owner, and no-mixed-generation evidence—not improvised in this guide.
|
||||
|
||||
## Readiness and troubleshooting status
|
||||
|
||||
Until the future procedure is implemented, do not diagnose PostgreSQL with ad hoc SQL, connection
|
||||
strings, or initialization scripts. The future sanitized runner-verification readiness artifact is
|
||||
the required PostgreSQL readiness authority after its bootstrap/TLS prerequisites pass.
|
||||
For local PGlite development, diagnose application behavior without introducing a PostgreSQL
|
||||
connection.
|
||||
|
||||
Non-database local services may be inspected with their ordinary local health/log tools. Those
|
||||
checks do not certify PostgreSQL, federated deployment, or production readiness.
|
||||
@@ -0,0 +1,518 @@
|
||||
# Mosaic Stack — Developer Guide
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Architecture Overview](#architecture-overview)
|
||||
2. [Local Development Setup](#local-development-setup)
|
||||
3. [Building and Testing](#building-and-testing)
|
||||
4. [Adding New Agent Tools](#adding-new-agent-tools)
|
||||
5. [Adding New MCP Tools](#adding-new-mcp-tools)
|
||||
6. [Database Schema and Migrations](#database-schema-and-migrations)
|
||||
7. [Claude Code Skill Bridge](#claude-code-skill-bridge)
|
||||
8. [API Endpoint Reference](#api-endpoint-reference)
|
||||
9. [Local Fleet Canary](./fleet-local-canary.md)
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
Mosaic Stack is a TypeScript monorepo managed with **pnpm workspaces** and
|
||||
**Turborepo**.
|
||||
|
||||
```
|
||||
mosaic-mono-v1/
|
||||
├── apps/
|
||||
│ ├── gateway/ # NestJS + Fastify API server
|
||||
│ └── web/ # Next.js 16 + React 19 web dashboard
|
||||
├── packages/
|
||||
│ ├── agent/ # Agent session types (shared)
|
||||
│ ├── auth/ # BetterAuth configuration
|
||||
│ ├── brain/ # Structured data layer (projects, tasks, missions)
|
||||
│ ├── cli/ # mosaic CLI and TUI (Ink)
|
||||
│ ├── coord/ # Mission coordination engine
|
||||
│ ├── db/ # Drizzle ORM schema, migrations, client
|
||||
│ ├── design-tokens/ # Shared design system tokens
|
||||
│ ├── log/ # Agent log ingestion and tiering
|
||||
│ ├── memory/ # Preference and insight storage
|
||||
│ ├── mosaic/ # Install wizard and bootstrap utilities
|
||||
│ ├── prdy/ # PRD wizard CLI
|
||||
│ ├── quality-rails/ # Code quality scaffolder CLI
|
||||
│ ├── queue/ # Valkey-backed task queue
|
||||
│ └── types/ # Shared TypeScript types
|
||||
├── docker/ # Dockerfile(s) for containerized deployment
|
||||
├── infra/ # Infrastructure configuration (for example, OTEL collector)
|
||||
├── docker-compose.yml # Local services (Postgres, Valkey, OTEL, Jaeger)
|
||||
└── CLAUDE.md # Project conventions for AI coding agents
|
||||
```
|
||||
|
||||
### Key Technology Choices
|
||||
|
||||
| Concern | Technology |
|
||||
| ----------------- | ---------------------------------------- |
|
||||
| API framework | NestJS with Fastify adapter |
|
||||
| Web framework | Next.js 16 (App Router), React 19 |
|
||||
| ORM | Drizzle ORM |
|
||||
| Database | PostgreSQL 17 + pgvector extension |
|
||||
| Auth | BetterAuth |
|
||||
| Agent harness | Pi SDK (`@mariozechner/pi-coding-agent`) |
|
||||
| Queue | Valkey 8 (Redis-compatible) |
|
||||
| Build | pnpm workspaces + Turborepo |
|
||||
| CI | Woodpecker CI |
|
||||
| Observability | OpenTelemetry → Jaeger |
|
||||
| Module resolution | NodeNext (ESM everywhere) |
|
||||
|
||||
### Module System
|
||||
|
||||
All packages use `"type": "module"` and NodeNext resolution. Import paths must
|
||||
include the `.js` extension even when the source file is `.ts`.
|
||||
|
||||
NestJS `@Inject()` decorators must be used explicitly because `tsx`/`esbuild`
|
||||
does not support `emitDecoratorMetadata`.
|
||||
|
||||
---
|
||||
|
||||
## Local Development Setup
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Node.js 20+
|
||||
- pnpm 9+
|
||||
- Docker and Docker Compose
|
||||
|
||||
### 1. Clone and Install Dependencies
|
||||
|
||||
```bash
|
||||
git clone <repo-url> mosaic-mono-v1
|
||||
cd mosaic-mono-v1
|
||||
pnpm install
|
||||
```
|
||||
|
||||
### 2. Use the local PGlite tier
|
||||
|
||||
The supported local tier is in-process PGlite and requires no PostgreSQL service. Leave
|
||||
`DATABASE_URL` unset for this route. Its default local configuration uses PGlite and performs no
|
||||
external database probe.
|
||||
|
||||
If a local queue service is useful, start only that non-PostgreSQL service:
|
||||
|
||||
```bash
|
||||
docker compose up -d valkey
|
||||
```
|
||||
|
||||
Do not use the current Compose PostgreSQL service: it mounts legacy `infra/pg-init` SQL and is
|
||||
not qualified for KBN-101. Start OTEL Collector or Jaeger individually only when needed and
|
||||
without starting PostgreSQL.
|
||||
|
||||
### 3. Gateway/Web local process (held)
|
||||
|
||||
Do not start the current Gateway or web process as a local PGlite route. Gateway first loads the
|
||||
daemon configuration and then project environment files without a tier guard; a pre-existing
|
||||
`DATABASE_URL` can select PostgreSQL, where current startup still reaches runtime DDL/migrations.
|
||||
Creating a root `.env` that omits `DATABASE_URL` does not make this safe, so neither a local
|
||||
credential file nor a web environment file is a current developer procedure.
|
||||
|
||||
PGlite remains the supported in-process data-layer implementation, and the optional Valkey command
|
||||
above remains safe because it does not start PostgreSQL. A safe Gateway/Web local procedure is held
|
||||
until KBN-101-02 rejects a daemon, inherited, root, or app-local PostgreSQL DSN and any non-local
|
||||
tier before connection or DDL; KBN-101-05 then supplies the production renderer/Vault process-exec
|
||||
or `LoadCredential` boundary.
|
||||
|
||||
### Held future procedure
|
||||
|
||||
PostgreSQL local and federated deployment are held until KBN-101-00 (external bootstrap),
|
||||
KBN-101-03 (runner), and KBN-101-05 (renderer-backed deployment) land. The following is the
|
||||
**held, non-operative future activation order with no current command authority**:
|
||||
|
||||
external bootstrap → TLS/roles → `mosaic-db-migrator --run` →
|
||||
`mosaic-db-migrator --verify` → Gateway/Compose readiness.
|
||||
|
||||
Neither current Compose nor this development guide authorizes PostgreSQL initialization SQL,
|
||||
manual DDL, or a pre-runner start.
|
||||
|
||||
### 5. Gateway/Web start (held)
|
||||
|
||||
No Gateway/Web start command is currently authorized for the local PGlite route. Do not use root
|
||||
`pnpm dev` as a workaround: it additionally starts configured integrations and cannot establish the
|
||||
required local-tier/DSN isolation. Resume this section only after KBN-101-02 provides its
|
||||
fail-closed local-startup evidence.
|
||||
|
||||
---
|
||||
|
||||
## Building and Testing
|
||||
|
||||
### TypeScript Typecheck
|
||||
|
||||
```bash
|
||||
pnpm typecheck
|
||||
```
|
||||
|
||||
Runs `tsc --noEmit` across all packages in dependency order via Turborepo.
|
||||
|
||||
### Lint
|
||||
|
||||
```bash
|
||||
pnpm lint
|
||||
```
|
||||
|
||||
Runs ESLint across all packages. Config is in `eslint.config.mjs` at the root.
|
||||
|
||||
### Format Check
|
||||
|
||||
```bash
|
||||
pnpm format:check
|
||||
```
|
||||
|
||||
Runs Prettier in check mode. To auto-fix:
|
||||
|
||||
```bash
|
||||
pnpm format
|
||||
```
|
||||
|
||||
### Tests
|
||||
|
||||
```bash
|
||||
pnpm test
|
||||
```
|
||||
|
||||
Runs Vitest across all packages. The workspace config is at
|
||||
`vitest.workspace.ts`.
|
||||
|
||||
### Build
|
||||
|
||||
```bash
|
||||
pnpm build
|
||||
```
|
||||
|
||||
Builds all packages and apps in dependency order.
|
||||
|
||||
### Pre-Push Gates (MANDATORY)
|
||||
|
||||
All three must pass before any push:
|
||||
|
||||
```bash
|
||||
pnpm format:check && pnpm typecheck && pnpm lint
|
||||
```
|
||||
|
||||
A pre-push hook enforces this mechanically.
|
||||
|
||||
### CI Publish Channels
|
||||
|
||||
Woodpecker `.woodpecker/publish.yml` keeps stable and integration-line artifacts separate:
|
||||
|
||||
| Source | npm packages | Gateway image |
|
||||
| --------------------------------- | ------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| `main` push/manual or release tag | committed package versions published to Gitea npm without changing the dist-tag workflow | `gateway:sha-<short>` plus `gateway:latest` on `main`, and the release tag on tag events |
|
||||
| `next` push/manual | CI-computed prereleases, `<target-stable>-next.<CI_PIPELINE_NUMBER>`, published with `npm publish --tag next` | `gateway:sha-<short>` only |
|
||||
|
||||
`next` never publishes npm `latest` or Docker `latest`. The next npm publish step verifies that `@mosaicstack/mosaic@next` resolves to the computed prerelease before the pipeline can pass.
|
||||
|
||||
---
|
||||
|
||||
## Adding New Agent Tools
|
||||
|
||||
Agent tools are Pi SDK `ToolDefinition` objects registered in
|
||||
`apps/gateway/src/agent/agent.service.ts`.
|
||||
|
||||
### 1. Create a Tool Factory File
|
||||
|
||||
Add a new file in `apps/gateway/src/agent/tools/`:
|
||||
|
||||
```typescript
|
||||
// apps/gateway/src/agent/tools/my-tools.ts
|
||||
import { Type } from '@sinclair/typebox';
|
||||
import type { ToolDefinition } from '@mariozechner/pi-coding-agent';
|
||||
|
||||
export function createMyTools(): ToolDefinition[] {
|
||||
const myTool: ToolDefinition = {
|
||||
name: 'my_tool_name',
|
||||
label: 'Human Readable Label',
|
||||
description: 'What this tool does.',
|
||||
parameters: Type.Object({
|
||||
input: Type.String({ description: 'The input parameter' }),
|
||||
}),
|
||||
async execute(_toolCallId, params) {
|
||||
const { input } = params as { input: string };
|
||||
const result = `Processed: ${input}`;
|
||||
return {
|
||||
content: [{ type: 'text' as const, text: result }],
|
||||
details: undefined,
|
||||
};
|
||||
},
|
||||
};
|
||||
|
||||
return [myTool];
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Register the Tools in AgentService
|
||||
|
||||
In `apps/gateway/src/agent/agent.service.ts`, import and call your factory
|
||||
alongside the existing tool registrations:
|
||||
|
||||
```typescript
|
||||
import { createMyTools } from './tools/my-tools.js';
|
||||
|
||||
// Inside the session creation logic where tools are assembled:
|
||||
const tools: ToolDefinition[] = [
|
||||
...createBrainTools(this.brain),
|
||||
...createCoordTools(this.coordService),
|
||||
...createMemoryTools(this.memory, this.embeddingService),
|
||||
...createFileTools(sandboxDir),
|
||||
...createGitTools(sandboxDir),
|
||||
...createShellTools(sandboxDir),
|
||||
...createWebTools(),
|
||||
...createMyTools(), // Add this line
|
||||
...mcpTools,
|
||||
...skillTools,
|
||||
];
|
||||
```
|
||||
|
||||
### 3. Export from the Tools Index
|
||||
|
||||
Add an export to `apps/gateway/src/agent/tools/index.ts`:
|
||||
|
||||
```typescript
|
||||
export { createMyTools } from './my-tools.js';
|
||||
```
|
||||
|
||||
### 4. Typecheck and Test
|
||||
|
||||
```bash
|
||||
pnpm typecheck
|
||||
pnpm test
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding New MCP Tools
|
||||
|
||||
Mosaic connects to external MCP servers via `McpClientService`. To expose tools
|
||||
from a new MCP server:
|
||||
|
||||
### 1. Run an MCP Server
|
||||
|
||||
Implement a standard MCP server that exposes tools via the streamable HTTP
|
||||
transport or SSE transport. The server must accept connections at a `/mcp`
|
||||
endpoint.
|
||||
|
||||
### 2. Gateway MCP configuration (held)
|
||||
|
||||
Do not configure MCP endpoint credentials, write them to a local environment file, or restart the
|
||||
Gateway from this guide. Gateway/Web startup is held until KBN-101-02 supplies fail-closed
|
||||
local-tier/DSN isolation and KBN-101-05 supplies the renderer/Vault process-exec or
|
||||
`LoadCredential` secret-consumer interface. The future authenticated MCP route requires verified
|
||||
HTTPS and certificate validation; plaintext bearer-token examples are forbidden.
|
||||
|
||||
### Tool Naming
|
||||
|
||||
Bridged MCP tool names are taken directly from the MCP server's tool manifest.
|
||||
Ensure names do not conflict with built-in tools (check
|
||||
`apps/gateway/src/agent/tools/`).
|
||||
|
||||
---
|
||||
|
||||
## Database Schema and Migrations
|
||||
|
||||
The schema lives in a single file:
|
||||
`packages/db/src/schema.ts`
|
||||
|
||||
### Schema Overview
|
||||
|
||||
| Table | Purpose |
|
||||
| -------------------- | ------------------------------------------------- |
|
||||
| `users` | User accounts (BetterAuth-compatible) |
|
||||
| `sessions` | Auth sessions |
|
||||
| `accounts` | OAuth accounts |
|
||||
| `verifications` | Email verification tokens |
|
||||
| `projects` | Project records |
|
||||
| `missions` | Mission records (linked to projects) |
|
||||
| `tasks` | Task records (linked to projects and/or missions) |
|
||||
| `conversations` | Chat conversation metadata |
|
||||
| `messages` | Individual chat messages |
|
||||
| `preferences` | Per-user key-value preference store |
|
||||
| `insights` | Vector-embedded memory insights |
|
||||
| `agent_logs` | Agent interaction logs (hot/warm/cold tiers) |
|
||||
| `skills` | Installed agent skills |
|
||||
| `summarization_jobs` | Log summarization job tracking |
|
||||
|
||||
The `insights` table uses a `vector(1536)` column (pgvector) for semantic search.
|
||||
|
||||
### PostgreSQL schema work (held)
|
||||
|
||||
Do not prepare or run a PostgreSQL target from this branch. The sole runner, bootstrap, and
|
||||
renderer are future KBN-101 artifacts, not current commands. When KBN-101-00/-03/-05 land, the
|
||||
owned activation documentation will require external bootstrap → TLS/roles → runner `--run` →
|
||||
runner `--verify` → Gateway/Compose readiness.
|
||||
|
||||
### Generating migration artifacts
|
||||
|
||||
`pnpm --filter @mosaicstack/db db:generate` is an offline artifact-generation command. It does
|
||||
not authorize connecting to or initializing PostgreSQL. A future reviewed PostgreSQL procedure
|
||||
will determine when its output is applied.
|
||||
|
||||
### Drizzle Config
|
||||
|
||||
Config is at `packages/db/drizzle.config.ts`. The schema file path and output directory are
|
||||
defined there.
|
||||
|
||||
### Adding a New Table
|
||||
|
||||
1. Add the table definition to `packages/db/src/schema.ts`.
|
||||
2. Export it from `packages/db/src/index.ts`.
|
||||
3. Generate the offline artifact with `pnpm --filter @mosaicstack/db db:generate`.
|
||||
4. Do not apply it to PostgreSQL until the future KBN-101 activation artifacts and their owned
|
||||
procedure are available. Direct schema push is not a production-like workflow.
|
||||
|
||||
---
|
||||
|
||||
## Claude Code Skill Bridge
|
||||
|
||||
The framework's canonical skill root is `~/.config/mosaic/skills/`; Claude Code
|
||||
requires registrations under `~/.claude/skills/`. The implementation in
|
||||
`packages/mosaic/src/commands/skill.ts` owns only direct-child symlinks whose
|
||||
resolved target remains inside the canonical root.
|
||||
|
||||
Security invariants:
|
||||
|
||||
1. Validate the user-supplied name before filesystem access against
|
||||
`[A-Za-z0-9][A-Za-z0-9._-]*`. Separators, control characters, whitespace,
|
||||
`..`, absolute paths, and leading `-` are invalid; filesystem-derived invalid
|
||||
names are escaped before terminal output.
|
||||
2. Never replace a real file, directory, foreign symlink, or live misdirected
|
||||
symlink in the Claude skill directory.
|
||||
3. Repair a dangling link only when its lexical target is inside the canonical
|
||||
Mosaic skills root.
|
||||
4. Unregister only a symlink pointing inside that root.
|
||||
5. Enumerate canonical directories at runtime; never hardcode framework skill
|
||||
names.
|
||||
|
||||
`finalizeStage` reconciles after wizard/framework synchronization, and
|
||||
`runFrameworkReseed` reconciles after the sync-only `mosaic update` path. A
|
||||
foreign conflict is reported but does not prevent unrelated canonical skills
|
||||
from registering. Filesystem tests use injected temporary roots in
|
||||
`skill.spec.ts`, `finalize-skills.spec.ts`, and `update-checker.reseed.spec.ts`.
|
||||
|
||||
M1 intentionally manages Claude Code only. Pi's Mosaic launcher can discover the
|
||||
canonical root directly. Codex still relies on the existing full skill-sync
|
||||
linker and needs separate parity analysis before this lifecycle API is extended.
|
||||
|
||||
## API Endpoint Reference
|
||||
|
||||
All endpoints are served by the gateway at `http://localhost:14242` by default.
|
||||
|
||||
### Authentication
|
||||
|
||||
Authentication uses BetterAuth session cookies. The auth handler is mounted at
|
||||
`/api/auth/*` via a Fastify low-level hook in
|
||||
`apps/gateway/src/auth/auth.controller.ts`.
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| ------------------------- | ------ | -------------------------------- |
|
||||
| `/api/auth/sign-in/email` | POST | Sign in with email/password |
|
||||
| `/api/auth/sign-up/email` | POST | Register a new account |
|
||||
| `/api/auth/sign-out` | POST | Sign out (clears session cookie) |
|
||||
| `/api/auth/get-session` | GET | Returns the current session |
|
||||
|
||||
### Chat
|
||||
|
||||
WebSocket namespace `/chat` (Socket.IO). Authentication via session cookie.
|
||||
|
||||
Events sent by the client:
|
||||
|
||||
| Event | Payload | Description |
|
||||
| --------- | --------------------------------------------------- | -------------- |
|
||||
| `message` | `{ content, conversationId?, provider?, modelId? }` | Send a message |
|
||||
|
||||
Events emitted by the server:
|
||||
|
||||
| Event | Payload | Description |
|
||||
| ------- | --------------------------- | ---------------------- |
|
||||
| `token` | `{ token, conversationId }` | Streaming token |
|
||||
| `end` | `{ conversationId }` | Stream complete |
|
||||
| `error` | `{ message }` | Error during streaming |
|
||||
|
||||
HTTP endpoints (`apps/gateway/src/chat/chat.controller.ts`):
|
||||
|
||||
| Endpoint | Method | Auth | Description |
|
||||
| -------------------------------------- | ------ | ---- | ------------------------------- |
|
||||
| `/api/chat/conversations` | GET | User | List conversations |
|
||||
| `/api/chat/conversations/:id/messages` | GET | User | Get messages for a conversation |
|
||||
|
||||
### Admin
|
||||
|
||||
All admin endpoints require `role = admin`.
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| --------------------------------- | ------ | -------------------- |
|
||||
| `GET /api/admin/users` | GET | List all users |
|
||||
| `GET /api/admin/users/:id` | GET | Get a single user |
|
||||
| `POST /api/admin/users` | POST | Create a user |
|
||||
| `PATCH /api/admin/users/:id/role` | PATCH | Update user role |
|
||||
| `POST /api/admin/users/:id/ban` | POST | Ban a user |
|
||||
| `POST /api/admin/users/:id/unban` | POST | Unban a user |
|
||||
| `DELETE /api/admin/users/:id` | DELETE | Delete a user |
|
||||
| `GET /api/admin/health` | GET | System health status |
|
||||
|
||||
### Agent / Providers
|
||||
|
||||
| Endpoint | Method | Auth | Description |
|
||||
| ------------------------------------ | ------ | ---- | ----------------------------------- |
|
||||
| `GET /api/agent/providers` | GET | User | List all providers and their models |
|
||||
| `GET /api/agent/providers/models` | GET | User | List available models |
|
||||
| `POST /api/agent/providers/:id/test` | POST | User | Test provider connectivity |
|
||||
|
||||
### Projects / Brain
|
||||
|
||||
| Endpoint | Method | Auth | Description |
|
||||
| -------------------------------- | ------ | ---- | ---------------- |
|
||||
| `GET /api/brain/projects` | GET | User | List projects |
|
||||
| `POST /api/brain/projects` | POST | User | Create a project |
|
||||
| `GET /api/brain/projects/:id` | GET | User | Get a project |
|
||||
| `PATCH /api/brain/projects/:id` | PATCH | User | Update a project |
|
||||
| `DELETE /api/brain/projects/:id` | DELETE | User | Delete a project |
|
||||
| `GET /api/brain/tasks` | GET | User | List tasks |
|
||||
| `POST /api/brain/tasks` | POST | User | Create a task |
|
||||
| `GET /api/brain/tasks/:id` | GET | User | Get a task |
|
||||
| `PATCH /api/brain/tasks/:id` | PATCH | User | Update a task |
|
||||
| `DELETE /api/brain/tasks/:id` | DELETE | User | Delete a task |
|
||||
|
||||
### Memory / Preferences
|
||||
|
||||
| Endpoint | Method | Auth | Description |
|
||||
| ----------------------------- | ------ | ---- | -------------------- |
|
||||
| `GET /api/memory/preferences` | GET | User | Get user preferences |
|
||||
| `PUT /api/memory/preferences` | PUT | User | Upsert a preference |
|
||||
|
||||
### MCP Server (Gateway-side)
|
||||
|
||||
| Endpoint | Method | Auth | Description |
|
||||
| ----------- | ------ | --------------------------------------------- | ----------------------------- |
|
||||
| `POST /mcp` | POST | User (session cookie or Authorization header) | MCP streamable HTTP transport |
|
||||
| `GET /mcp` | GET | User | MCP SSE stream reconnect |
|
||||
|
||||
### Skills
|
||||
|
||||
| Endpoint | Method | Auth | Description |
|
||||
| ------------------------ | ------ | ----- | --------------------- |
|
||||
| `GET /api/skills` | GET | User | List installed skills |
|
||||
| `POST /api/skills` | POST | Admin | Install a skill |
|
||||
| `PATCH /api/skills/:id` | PATCH | Admin | Update a skill |
|
||||
| `DELETE /api/skills/:id` | DELETE | Admin | Remove a skill |
|
||||
|
||||
### Coord (Mission Coordination)
|
||||
|
||||
| Endpoint | Method | Auth | Description |
|
||||
| ------------------------------- | ------ | ---- | ---------------- |
|
||||
| `GET /api/coord/missions` | GET | User | List missions |
|
||||
| `POST /api/coord/missions` | POST | User | Create a mission |
|
||||
| `GET /api/coord/missions/:id` | GET | User | Get a mission |
|
||||
| `PATCH /api/coord/missions/:id` | PATCH | User | Update a mission |
|
||||
|
||||
### Observability
|
||||
|
||||
OpenTelemetry traces are exported to the OTEL collector (`OTEL_EXPORTER_OTLP_ENDPOINT`).
|
||||
View traces in Jaeger at `http://localhost:16686`.
|
||||
|
||||
Tracing is initialized before NestJS bootstrap in
|
||||
`apps/gateway/src/tracing.ts`. The import order in `apps/gateway/src/main.ts`
|
||||
is intentional: `import './tracing.js'` must come before any NestJS imports.
|
||||
@@ -0,0 +1,177 @@
|
||||
# Local Fleet Canary
|
||||
|
||||
The local fleet canary runs a small tmux-backed Mosaic agent fleet on an
|
||||
isolated tmux socket. The default socket is `mosaic-fleet`; the commands do
|
||||
not use or stop the default tmux server.
|
||||
|
||||
## Files
|
||||
|
||||
Product-owned defaults:
|
||||
|
||||
- `packages/mosaic/framework/fleet/roster.schema.json`
|
||||
- `packages/mosaic/framework/fleet/examples/minimal.yaml`
|
||||
- `packages/mosaic/framework/fleet/examples/local-canary.yaml`
|
||||
- `packages/mosaic/framework/systemd/user/mosaic-tmux-holder.service`
|
||||
- `packages/mosaic/framework/systemd/user/[email protected]`
|
||||
- `packages/mosaic/framework/tools/fleet/start-agent-session.sh`
|
||||
- `packages/mosaic/framework/tools/tmux/agent-send.sh`
|
||||
- `packages/mosaic/framework/tools/tmux/send-message.sh`
|
||||
|
||||
These files are published through `packages/mosaic/package.json`, whose `files`
|
||||
allowlist includes `framework` along with `dist`.
|
||||
|
||||
Site-owned local roster:
|
||||
|
||||
```text
|
||||
~/.config/mosaic/fleet/roster.yaml
|
||||
```
|
||||
|
||||
Do not put a host-specific full roster into product defaults. Start from an
|
||||
example and edit the local roster after `mosaic fleet init --write`.
|
||||
|
||||
## Install
|
||||
|
||||
Minimal canary:
|
||||
|
||||
```bash
|
||||
mosaic fleet init --profile minimal --write
|
||||
# If a site-owned roster already exists, inspect it first; overwrite only explicitly:
|
||||
# mosaic fleet init --profile minimal --write --force
|
||||
mosaic fleet install-systemd
|
||||
systemctl --user daemon-reload
|
||||
mosaic fleet start
|
||||
mosaic fleet verify
|
||||
```
|
||||
|
||||
Small dogfood roster:
|
||||
|
||||
```bash
|
||||
mosaic fleet init --profile local-canary --write
|
||||
# Use --force only after preserving any site-owned roster changes.
|
||||
mosaic fleet install-systemd
|
||||
systemctl --user daemon-reload
|
||||
mosaic fleet start
|
||||
mosaic fleet status
|
||||
```
|
||||
|
||||
## Agent Operations
|
||||
|
||||
```bash
|
||||
mosaic agent roster
|
||||
mosaic agent status
|
||||
mosaic agent status canary-pi
|
||||
mosaic agent send canary-pi --message "status check"
|
||||
mosaic agent reset canary-pi --new
|
||||
mosaic agent tail canary-pi -n 80
|
||||
```
|
||||
|
||||
These commands read the roster and target the configured tmux socket. The
|
||||
generated systemd agent services use `start-agent-session.sh`; message delivery
|
||||
uses the tmux send tools with `-L mosaic-fleet`.
|
||||
|
||||
`mosaic agent send` is operator-origin traffic unless a caller explicitly says
|
||||
otherwise. The CLI always passes a deterministic source label to
|
||||
`agent-send.sh` with `-S`, defaulting to `<hostname>:operator`, so it does not
|
||||
query the target tmux socket and accidentally identify as an active agent pane.
|
||||
Use `--source-label <label>` or `--source <label>` only when deliberately
|
||||
impersonating a known handoff lane. The lower-level inter-agent wrapper
|
||||
`agent-send.sh -S <label>` remains the explicit source override for scripts.
|
||||
|
||||
## Verification
|
||||
|
||||
Use these checks before expanding the roster:
|
||||
|
||||
```bash
|
||||
tmux -L mosaic-fleet ls
|
||||
tmux ls
|
||||
mosaic fleet verify
|
||||
systemctl --user status mosaic-tmux-holder.service
|
||||
```
|
||||
|
||||
Expected results:
|
||||
|
||||
- `tmux -L mosaic-fleet ls` shows `_holder` and roster agent sessions.
|
||||
- `tmux ls` shows only the default tmux server sessions and is not changed by
|
||||
fleet start/stop operations.
|
||||
- `mosaic fleet verify` checks exact session targets on the isolated socket.
|
||||
- `systemctl --user status ...` may show `active (exited)` for oneshot units;
|
||||
that means the unit ran, not that an agent pane is live. Treat tmux
|
||||
`has-session`, `list-panes`, process tree, and logs as the liveness evidence.
|
||||
|
||||
## Recovery — rebuild generated env projections
|
||||
|
||||
Each agent's `~/.config/mosaic/fleet/agents/<name>.env.generated` is a
|
||||
deterministic projection of `roster.yaml` (the SSOT) that the launcher
|
||||
(`start-agent-session.sh`) sources at start. If an upgrade or a manual mistake
|
||||
wipes or diverges those projections, rebuild them from the roster with
|
||||
`mosaic fleet regen` — do NOT restart the affected unit first.
|
||||
|
||||
```bash
|
||||
mosaic fleet regen # dry-run (default): show create/rebuild plan per agent
|
||||
mosaic fleet regen --json # same plan, machine-readable
|
||||
mosaic fleet regen --write # rebuild fleet/agents/<name>.env.generated on disk
|
||||
```
|
||||
|
||||
`regen` is projection-only and **never restarts an agent** — it has no path to
|
||||
systemd lifecycle. It is dry-run by default, deterministic/idempotent, uses the
|
||||
same roster→env mapping as `mosaic fleet reconcile`, and emits paths and counts
|
||||
only (never the projected `KEY=value` body). After `--write`, verify each unit
|
||||
resolves the intended values before restarting one unit at a time. The unit sets
|
||||
no `EnvironmentFile=` — `start-agent-session.sh` sources `.env.generated` itself —
|
||||
so verify the generated file directly and the launcher path, not a nonexistent
|
||||
`EnvironmentFile` property:
|
||||
|
||||
```bash
|
||||
test -f ~/.config/mosaic/fleet/agents/<name>.env.generated
|
||||
systemctl --user cat mosaic-agent@<name> | grep ExecStart
|
||||
systemctl --user restart mosaic-agent@<name>
|
||||
```
|
||||
|
||||
Full recovery runbook and the three-layer #791 protection model (manifest
|
||||
ownership → pre-update snapshot/restore → regen): see
|
||||
[Upgrade Safety & Recovery](./upgrade-safety-and-recovery.md).
|
||||
|
||||
## Release Preflight
|
||||
|
||||
Run this checklist before cutting or dogfooding a fleet release:
|
||||
|
||||
- Real AI dogfood: send at least one task through `mosaic agent send`, then
|
||||
confirm the agent accepted/responded using pane, process, or log evidence.
|
||||
- Restart/stop/idempotency: run `mosaic fleet start`, `restart`, `stop`, and a
|
||||
repeated `start` against the named socket; verify the default tmux server is
|
||||
unchanged.
|
||||
- Liveness verification: run `mosaic fleet verify` and confirm roster sessions
|
||||
with `tmux -L mosaic-fleet ls` or exact `has-session` checks.
|
||||
- Package dry-run: run `npm pack --dry-run --json` from `packages/mosaic` and
|
||||
confirm `framework/fleet`, `framework/systemd/user`,
|
||||
`framework/tools/fleet`, and `framework/tools/tmux` assets are included.
|
||||
- Mosaic update test: install or upgrade from the packed artifact in a temporary
|
||||
Mosaic home and confirm `mosaic update` or the release upgrade path does not
|
||||
remove local roster/config files.
|
||||
|
||||
## Rollback
|
||||
|
||||
Stop the local canary:
|
||||
|
||||
```bash
|
||||
mosaic fleet stop
|
||||
systemctl --user disable mosaic-agent@canary-pi.service
|
||||
systemctl --user disable mosaic-tmux-holder.service
|
||||
systemctl --user daemon-reload
|
||||
```
|
||||
|
||||
For a full local cleanup of generated canary files:
|
||||
|
||||
```bash
|
||||
rm -f ~/.config/systemd/user/[email protected]
|
||||
rm -f ~/.config/systemd/user/mosaic-tmux-holder.service
|
||||
rm -rf ~/.config/mosaic/fleet
|
||||
rm -rf ~/.config/mosaic/tools/fleet
|
||||
```
|
||||
|
||||
This rollback leaves the default tmux server untouched. If a canary session is
|
||||
still present after service stop, remove only the isolated socket server:
|
||||
|
||||
```bash
|
||||
tmux -L mosaic-fleet kill-server
|
||||
```
|
||||
@@ -0,0 +1,44 @@
|
||||
# Lease broker operations
|
||||
|
||||
Place the socket and state file in a dedicated directory with mode `0700`. Start the packaged daemon with:
|
||||
|
||||
```bash
|
||||
python3 "$MOSAIC_HOME/tools/lease-broker/daemon.py" \
|
||||
--socket /run/user/1000/mosaic-lease/broker.sock \
|
||||
--state /run/user/1000/mosaic-lease/state.json
|
||||
```
|
||||
|
||||
The broker refuses an existing parent directory whose mode is not exactly `0700`, an existing state file not at `0600`, corrupt/incompatible state, or an already-existing socket path. After bind it sets the socket to `0600`. It never silently unlinks a pre-existing socket. On normal termination it unlinks only the socket inode it created, so it does not remove a replacement path.
|
||||
|
||||
Before launching Claude, Claudex, or Pi, export the socket path; `mosaic` then runs the runtime through the packaged register-and-exec wrapper:
|
||||
|
||||
```bash
|
||||
export MOSAIC_LEASE_BROKER_SOCKET=/run/user/1000/mosaic-lease/broker.sock
|
||||
mosaic claude # or: mosaic claudex, mosaic yolo claudex, mosaic pi
|
||||
```
|
||||
|
||||
The wrapper obtains a broker-minted session ID, creates a private `generation-<session>.state` file beside the socket, and `exec`s the runtime without changing its PID/starttime anchor. The all-tools Claude `PreToolUse` hook and Pi `tool_call` handler inherit that identity and read the current generation from the file. Claudex retains its isolated proxy environment and config directory; Mosaic merges the mandatory all-tools and compaction-lifecycle hooks into that isolated `settings.json` before invoking the same wrapper. PRDY init/update, QA remediation, coord, orchestrator, and fleet launchers also converge on this boundary. Broker registration failure, unsafe isolated settings, unsafe generation state, or missing identity denies launch/tool execution fail-closed; broker timeout/unavailability and malformed replies also block tools.
|
||||
|
||||
Claude `PreCompact` and `SessionStart(compact)` hooks and Pi pre-/post-compaction handlers invoke `revoke-lease.py`. Pi `session_start` reload/new/resume/fork and Claude resume/clear advance the locked generation before revocation, so a replacement session inherits no lease even when PID/starttime stay unchanged. Do not invoke the revoker manually as a way to restore authority; it only removes authority. If a lifecycle hook reports failure, stop consequential work and repair broker/generation-state availability before re-verification.
|
||||
|
||||
Run the permanent launch inventory locally with:
|
||||
|
||||
```bash
|
||||
python3 packages/mosaic/framework/tools/lease-broker/check-runtime-launches.py --root .
|
||||
```
|
||||
|
||||
The same check runs in the Mosaic package test suite and therefore in root CI. Any direct Claude/Pi binary launch must be replaced with `launch-runtime.py`, `execLeaseGatedRuntime`, or the gated `mosaic` runtime command; do not add static allowlist exceptions.
|
||||
|
||||
Clients must complete the request boundary before waiting for a reply. After sending the single JSON object and its terminating newline, the client **MUST half-close the socket's write side** (`shutdown(SHUT_WR)` in POSIX clients; `socket.end()` in Node) and only then await the response. Merely calling `write()` and waiting is invalid: the broker waits for EOF to enforce the exact-one-frame contract and fails closed at its one-second deadline. Do not replace `end()` with `write()` in client helpers. A delayed second frame remains malformed and is rejected.
|
||||
|
||||
`mosaic_context_recover` is the only unverified mutator class. Its durable `mosaic-context-refresh` skill is a thin wrapper over `tools/lease-broker/recover-context.py`: `begin` has the broker rebuild the validated `B_payload`/`H_payload`, revoke first, and mint a new `PENDING_DELIVERY` receipt challenge; `complete` accepts neither receipt text nor a challenge argument. Claude maps only the exact direct recovery executable/validated arguments to this exempt tool identity; ordinary `Bash` remains gated. Pi exposes only the `mosaic_context_recover` custom tool; ordinary `bash` and all other tools remain gated. A normal-path receipt cannot be replayed through recovery because each retry begins a distinct recovery cycle and recovery completion cannot receive caller-presented evidence.
|
||||
|
||||
Production daemon startup creates a separate private observer socket unless a test-only `--test-observer-file` fixture is selected. Claude's Stop hook sends its exact latest assistant entry and Pi's `message_end` handler sends only finalized assistant content to that authenticated transport; the broker public socket never accepts message text. This is byte-build and private out-of-process harness wiring only: do not activate it against a live daemon, live socket, systemd service, tmux session, or model-output stream outside the controlled integration procedure.
|
||||
|
||||
Receipt honesty is load-bearing: absent, malformed, prefix-truncated, and observable adapter-mutated terminal receipts do not promote. A tail-only case is non-promoting only where the concrete terminal payload is malformed or observably incomplete. A tail-preserving middle drop is **not receipt-detectable**; it is the disclosed T-C injection-contract residual deferred to WI-7 server-side evidence. The receipt remains a T-A delivery/liveness prerequisite, never a safety, obedience, or residency proof. The framework skill is source-resident and bridge-projected on install/upgrade; do not hand-create a live runtime symlink.
|
||||
|
||||
After a runtime exits, its `generation-<session>.state` file may be removed only after verifying that no process for that broker-minted session remains; stale files carry no lease authority but should be retained during incident analysis. After a broker crash, preserve the protected state file and restart only after verifying that no broker owns the socket. Restart intentionally clears all volatile VERIFIED leases. A leftover socket requires an operator to verify the owning service is stopped and remove that exact socket deliberately. Corrupt, oversized, symlinked, or non-regular state fails closed; do not overwrite it. Preserve it for incident review and establish new state only through an explicit operational decision, which invalidates prior sessions and tokens.
|
||||
|
||||
## Security posture
|
||||
|
||||
Directory `0700` plus socket/state `0600` is built-in same-principal hardening only: it excludes other UIDs but does **not** stop the same UID from unlinking and counterfeiting the socket. It therefore does not close T-C same-UID replacement. WI-1 does not provide a distinct-principal boundary. A stronger distinct-principal deployment requires an external protected proxy, ACL, or service boundary that clients cannot unlink or rebind and that preserves the authenticated client identity required by the broker's `SO_PEERCRED` and ancestry checks. Server-side branch protection remains the irreducible backstop.
|
||||
@@ -0,0 +1,98 @@
|
||||
# Migrating to the Federated Tier
|
||||
|
||||
> **KBN-101-07 ownership:** This active documentation is a **non-operative KBN-101
|
||||
> contract** with no current command authority until KBN-101-00, KBN-101-02, KBN-101-03, KBN-101-05, and KBN-101-06 land and
|
||||
> KBN-101-08 activates an exact reviewed release. The commands below describe the produced interface only. Do not run them on the
|
||||
> current branch or replace them with direct PostgreSQL, raw SQL, legacy storage migration, or
|
||||
> credential-on-argv procedures.
|
||||
|
||||
## Held future procedure
|
||||
|
||||
This section is non-operative and grants no current command authority until KBN-101-00, KBN-101-03, and KBN-101-05 land.
|
||||
|
||||
The deployment control plane executes the complete held future procedure, in order: external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness. The
|
||||
runner is the only attestation producer after its verified TLS, identity, manifest, and schema
|
||||
checks. A data importer is never a schema bootstrap, extension installer, repair command, or DDL
|
||||
consumer.
|
||||
|
||||
## Target material contract
|
||||
|
||||
KBN-101-05 obtains the target URL from Vault KV-v2
|
||||
`secret-{env}/mosaic-stack/database/importer`, key `url`, and reads its authenticated version from
|
||||
the same successful response `data.metadata.version`. A hash or DSN byte sequence is not a
|
||||
provider version. The renderer treats URL bytes and provider version as one generation, writes a
|
||||
temporary generation directory with fsync plus atomic rename, and creates separate immutable
|
||||
consumer mounts. Swarm uses distinct versioned secret/config references. A deployment cannot mix
|
||||
generations.
|
||||
|
||||
| Consumer | Permitted material |
|
||||
| --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Migrator-attestation producer (`10003:10003`) | Its own migration URL/CA; read-only `/run/secrets/mosaic-migrate-target-url` and `/run/secrets/mosaic-migrate-target-version`, each `0400`, solely to bind; producer-only attestation output at `/run/mosaic-attestations-producer/migrate-target.v1.json`; root-wrapper-only signing key. It never connects with, uses, exports, logs, or forwards the importer URL/version. |
|
||||
| Privileged deployment handoff controller | After runner success and before importer creation, it receives only root-owned non-secret expected provider-version/URL-SHA-256/generation descriptor and pinned public verifier key—not URL bytes or private key. It safe-opens/verifies descriptor and producer artifact, copies exact bytes to a new importer-only mount with fsync/atomic rename, sets `10002:10002` `0400`, seals it read-only, and refuses importer start on any partial/wrong-generation/wrong-owner/mode result. |
|
||||
| Importer (`10002:10002`) | Its own immutable `0400` copies at the same URL/version paths; CA at exact `DATABASE_TLS_CA_CERT_PATH=/run/secrets/mosaic-db-ca.crt`; pinned Ed25519 public key; read-only `/run/mosaic-attestations/migrate-target.v1.json` supplied only by the sealed handoff. |
|
||||
| Gateway/runtime/unrelated container | No importer URL/version, importer artifact, attestation private key, or unrelated CA mount. |
|
||||
|
||||
The migrator and importer safe-open URL, provider-version, attestation, and public-key files only
|
||||
with `O_RDONLY|O_CLOEXEC|O_NOFOLLOW`; they validate from the opened fd that the file is regular,
|
||||
has its expected owner/mode and link count one. The migrator digests only that URL fd for binding,
|
||||
then zeroizes/closes it. The importer reads URL bytes once into protected memory, validates the
|
||||
signed binding and exact CA before connecting from those same bytes, then zeroizes/closes every
|
||||
fd. It neither logs nor exposes a URL/version/attestation/key oracle.
|
||||
|
||||
## Produced command interface
|
||||
|
||||
After activation and only after approved target preparation, the future interface is:
|
||||
|
||||
```bash
|
||||
# Deployment control plane has already completed the held runner procedure above.
|
||||
mosaic storage migrate-tier --to federated \
|
||||
--target-url-file /run/secrets/mosaic-migrate-target-url \
|
||||
--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json \
|
||||
--dry-run
|
||||
```
|
||||
|
||||
The provider-version file is fixed deployment material, not argv. This connecting dry-run consumes its nonce; before an actual copy, the deployment control plane must provide fresh runner verification and a new sealed handoff. The runner uses its migration
|
||||
identity; the importer connects only as non-DDL `mosaic_data_importer` and only after all
|
||||
pre-connect validation. After verified TLS and before DML it compares PostgreSQL system ID,
|
||||
database OID, `current_user`, CA/SPKI, and manifest/schema fingerprints to the artifact.
|
||||
|
||||
## Required refusals and evidence
|
||||
|
||||
KBN-101-02/-03/-05/-06 must prove, with stable sanitized errors, that no target connection occurs
|
||||
for missing/unsafe URL/version/attestation/public-key files; symlink, hardlink, owner, mode, or
|
||||
TOCTOU violations; mixed URL/version generations; missing/wrong CA mount; stale/replayed/tampered
|
||||
or revoked-key artifacts; provider rotation/revocation; wrong TLS/server/database/role/manifest
|
||||
binding; raw `--target-url`; `DATABASE_URL` fallback; runtime/owner identity; consumer leakage;
|
||||
or any DDL attempt. Post-connect identity mismatch closes with zero DML/DDL. Tests also prove no
|
||||
forwarding, child environment, logging, or error oracle leaks URL/version/key/artifact contents.
|
||||
|
||||
The attestation is credential-free JCS with detached Ed25519 signature and binds issued/expiry,
|
||||
nonce, authenticated provider version, exact URL-fd SHA-256, TLS host/port/database, CA/SPKI,
|
||||
PostgreSQL system ID/database OID, importer role, manifest/schema, and producer identity. Provider
|
||||
version rotation invalidates an old artifact and requires a fresh rendered generation plus runner
|
||||
verification.
|
||||
|
||||
## Actual copy after dry-run
|
||||
|
||||
After reviewed dry-run, obtain the required fresh verification/attestation generation, then use:
|
||||
|
||||
```bash
|
||||
# Deployment control plane has supplied fresh runner verification and attestation.
|
||||
mosaic storage migrate-tier --to federated \
|
||||
--target-url-file /run/secrets/mosaic-migrate-target-url \
|
||||
--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json \
|
||||
--yes
|
||||
```
|
||||
|
||||
The dry-run artifact is terminally replayed and must be rejected; `--yes` bypasses no file,
|
||||
generation, signature, TLS, identity, or DDL control.
|
||||
|
||||
## Data boundary and recovery
|
||||
|
||||
The importer has only an allowlisted mutable-table DML registry. It has no grant for immutable KBN
|
||||
relations, schemas, roles, memberships, extensions, catalogs, or the Drizzle ledger. Source PGlite
|
||||
uses its explicit local directory and does not make a PostgreSQL URL fallback valid.
|
||||
|
||||
A failed or ambiguous migration is a control-plane incident: preserve sanitized evidence, retain
|
||||
the approved backup/rollback state, and retry only after independent review. Never inspect,
|
||||
unlock, repair, or initialize the target with ad hoc SQL or copied credentials.
|
||||
@@ -0,0 +1,43 @@
|
||||
# Mos Connector Lease Operations — M1
|
||||
|
||||
## Operational status
|
||||
|
||||
M1 installs the durable schema and gateway policy/adapter boundary. It does **not** activate a connector, expose a lease administration endpoint, or cut over a channel. The default gateway connector-lease policy is deny-all until a later work package supplies an authorized server-side policy and concrete adapter.
|
||||
|
||||
## Events to monitor
|
||||
|
||||
Use correlation IDs to follow `connector_lease_audit_log` events:
|
||||
|
||||
| Event | Meaning |
|
||||
| ---------- | --------------------------------------------------------------------- |
|
||||
| `acquire` | First holder inserted for an unused binding |
|
||||
| `renew` | Current holder heartbeat extended the TTL |
|
||||
| `takeover` | Authorized CAS replaced the holder and incremented epoch |
|
||||
| `release` | Current holder explicitly relinquished authority |
|
||||
| `expiry` | An expired current lease was observed |
|
||||
| `reject` | Policy, CAS, expiry, scope, or fencing validation denied an operation |
|
||||
|
||||
Audit data is metadata-only. Raw grant objects, connector payloads, scopes, tokens, approval references, and credentials must never be added to audit output.
|
||||
|
||||
## Incident checks
|
||||
|
||||
For suspected duplicate/stale connector effects:
|
||||
|
||||
1. Correlate the attempted operation with its `reject`, `takeover`, or `expiry` event.
|
||||
2. Compare the current row's connector ID, lease UUID, epoch, expiry, and release time with the adapter's normalized execution context.
|
||||
3. Treat an old epoch, old lease UUID, expired lease, or released lease as non-authoritative. Do not retry it as the old holder.
|
||||
4. Recovery uses the authorized takeover path with the observed expected epoch. Ordinary acquire is intentionally rejected for expired/released rows.
|
||||
5. If an external effect may already have happened, preserve evidence and do not assume lease fencing provides exactly-once replay safety.
|
||||
|
||||
## Migration and rollback safety
|
||||
|
||||
Migration `0016_salty_morlocks.sql` is additive: it creates two new tables and indexes without modifying existing authorization/session tables. Before rollout, normal database backup and migration verification still apply. Rolling application code back leaves unused additive tables in place; dropping tables is not part of automated rollback because it would destroy lease/audit evidence.
|
||||
|
||||
## Security constraints
|
||||
|
||||
- Tenant comes from authenticated gateway context, never a connector request field.
|
||||
- Logical agent, binding, connector, and scope identifiers use normalized constrained forms.
|
||||
- Takeover requires explicit gateway policy authorization and an expected epoch.
|
||||
- Default defense-in-depth TTL caps are 5 minutes for leases and 30 seconds for grants; policy may enforce stricter limits.
|
||||
- Validation and rejection audit complete before adapter side effects.
|
||||
- Existing authz and exact-action approval controls remain additional required gates; a valid connector lease does not bypass them.
|
||||
@@ -0,0 +1,147 @@
|
||||
# Upgrade Safety & Recovery
|
||||
|
||||
How Mosaic protects operator-owned configuration under `~/.config/mosaic` across
|
||||
framework upgrades, and how to recover if a projection is ever lost.
|
||||
|
||||
A framework upgrade runs `install.sh` in keep-mode (`MOSAIC_INSTALL_MODE=keep`,
|
||||
`MOSAIC_SYNC_ONLY=1`) to refresh framework-owned files in place. The incident
|
||||
this hardening addresses: an upgrade that silently overwrites or deletes a file
|
||||
the operator owns — credentials, personas, a roster, or a generated agent env —
|
||||
with no snapshot to fall back to.
|
||||
|
||||
Protection is layered. Each layer is independent; a later layer catches what an
|
||||
earlier one misses.
|
||||
|
||||
## Layer 1 — Manifest-owned sync (prevention)
|
||||
|
||||
The single source of truth for ownership is
|
||||
[`framework-manifest.txt`](../../packages/mosaic/framework/framework-manifest.txt).
|
||||
Both the bash installer and the TypeScript sync path resolve every path against
|
||||
this one file (parity is enforced by test), so they can never drift.
|
||||
|
||||
- Ownership is **allow-list, deny-wins**: a path is framework-owned only if a
|
||||
`[framework]` glob matches and no `[operator]` carve-out overrides it.
|
||||
- **Unknown paths default to operator** (fail-safe): a file the manifest never
|
||||
anticipated is treated as operator-owned and is never pruned.
|
||||
- Keep-mode does a non-deleting copy plus an explicit, manifest-scoped prune that
|
||||
only ever iterates framework globs — operator and unknown paths are
|
||||
structurally unreachable by the prune.
|
||||
|
||||
Result: a correct upgrade cannot touch operator config at all.
|
||||
|
||||
## Layer 2 — Durable pre-update snapshot + verify net (safety + rollback)
|
||||
|
||||
Before **any** mutation, the installer snapshots the operator-owned surface that
|
||||
exists into:
|
||||
|
||||
```
|
||||
${XDG_STATE_HOME:-~/.local/state}/mosaic/backups/pre-update-<UTC-timestamp>/
|
||||
```
|
||||
|
||||
- `0700` directories / `0600` files (`umask 077`, scoped and restored),
|
||||
outside `~/.config/mosaic` and outside any repo.
|
||||
- **Fail-open**: a snapshot failure warns but never aborts the upgrade it
|
||||
protects.
|
||||
- Retention is `MOSAIC_BACKUP_RETENTION` snapshots (default 5).
|
||||
|
||||
After the sync, a **verify net** compares each snapshot file against its target
|
||||
and restores (with a loud warning) any operator file the upgrade diverged or
|
||||
removed — a divergence means a manifest bug slipped through Layer 1.
|
||||
|
||||
Inspect and restore snapshots with the CLI:
|
||||
|
||||
```bash
|
||||
mosaic restore --list # dry-run: enumerate snapshots by timestamp
|
||||
mosaic restore --from <UTC-timestamp> # restore the operator surface from one snapshot
|
||||
mosaic restore --from <ts> --dry-run # preview a specific restore without writing
|
||||
```
|
||||
|
||||
`mosaic restore` reports **counts and relative paths only** — it never emits file
|
||||
contents, so a secret in `tools/_lib/credentials.json` is never echoed. Restores
|
||||
are confirmation-gated (`--yes` or `MOSAIC_ASSUME_YES`) and write each leaf
|
||||
atomically with `O_NOFOLLOW` (a symlink swapped in after the snapshot fails
|
||||
closed rather than following out of the managed tree).
|
||||
|
||||
## Layer 3 — Regeneration from roster SSOT (recovery)
|
||||
|
||||
Some operator files are **derived** and do not need a byte-for-byte snapshot to
|
||||
recover — they can be rebuilt from their source of truth. The fleet's per-agent
|
||||
generated env projections are the prime case:
|
||||
|
||||
- `~/.config/mosaic/fleet/agents/<name>.env.generated` is a deterministic
|
||||
projection of `~/.config/mosaic/fleet/roster.yaml`.
|
||||
- The launcher (`start-agent-session.sh`, invoked by
|
||||
`mosaic-agent@<name>.service`) sources that generated projection to establish
|
||||
each agent's identity, runtime, model, and working directory. If it is missing
|
||||
or wrong, the agent cannot launch with its intended identity.
|
||||
|
||||
`mosaic fleet regen` rebuilds those projections from the roster SSOT:
|
||||
|
||||
```bash
|
||||
mosaic fleet regen # dry-run (default): show what would be rebuilt
|
||||
mosaic fleet regen --json # same, machine-readable
|
||||
mosaic fleet regen --write # rebuild the projections on disk
|
||||
```
|
||||
|
||||
- **Dry-run by default.** Nothing is written until you pass `--write`.
|
||||
- **Deterministic and idempotent** — the projection is a pure function of the
|
||||
roster, so repeated `--write` runs produce byte-identical files.
|
||||
- **Projection-only. It never restarts an agent.** Recovery order forbids
|
||||
restart-before-verify; `regen` has no path to systemd lifecycle at all.
|
||||
- **It rebuilds only `<name>.env.generated`** — it never writes, relocates, or
|
||||
deletes the operator-owned `.env` / `.env.local` surface.
|
||||
- It **validates the roster the same way `reconcile` does** (persona resolution
|
||||
and protected-class tool-policy match), so a hand-edited or corrupt roster is
|
||||
rejected rather than projected, and a `--write` takes the shared reconcile
|
||||
lock so it cannot race a concurrent reconcile.
|
||||
- Output is **paths and counts only** — the rendered `KEY=value` body is never
|
||||
echoed.
|
||||
|
||||
`regen` uses the exact same roster→env mapping as `mosaic fleet reconcile`, so a
|
||||
recovered projection matches what a normal reconcile would have written.
|
||||
|
||||
## Recovery runbook — wiped `fleet/agents/*.env.generated`
|
||||
|
||||
If an upgrade (or a manual mistake) has left an agent without its generated
|
||||
projection, **do not restart the unit first** — a launch against a missing
|
||||
projection fails closed, and any stale state must be corrected before restart,
|
||||
not after.
|
||||
|
||||
1. **Prefer a snapshot restore if one exists** (byte-exact operator state):
|
||||
|
||||
```bash
|
||||
mosaic restore --list
|
||||
mosaic restore --from <UTC-timestamp>
|
||||
```
|
||||
|
||||
2. **Otherwise regenerate the derived projections from the roster SSOT:**
|
||||
|
||||
```bash
|
||||
mosaic fleet regen # confirm the plan (create vs rebuild per agent)
|
||||
mosaic fleet regen --write # rebuild fleet/agents/<name>.env.generated
|
||||
```
|
||||
|
||||
3. **Verify each unit will resolve the intended runtime/workdir _before_ any
|
||||
restart.** The unit sets **no** `EnvironmentFile=` — it launches from a minimal
|
||||
environment and `start-agent-session.sh` sources `.env.generated` itself, so
|
||||
verify the generated file directly and confirm the launcher path:
|
||||
|
||||
```bash
|
||||
# Confirm fleet/agents/<name>.env.generated exists and carries the intended
|
||||
# MOSAIC_AGENT_* values (name, runtime, model, workdir, socket).
|
||||
test -f ~/.config/mosaic/fleet/agents/<name>.env.generated
|
||||
# Confirm the unit launches the session script that reads it.
|
||||
systemctl --user cat mosaic-agent@<name> | grep ExecStart
|
||||
```
|
||||
|
||||
4. **Only then restart, one unit at a time:**
|
||||
|
||||
```bash
|
||||
systemctl --user restart mosaic-agent@<name>
|
||||
```
|
||||
|
||||
## See also
|
||||
|
||||
- Design: [`docs/design/791-upgrade-config-protection.md`](../design/791-upgrade-config-protection.md)
|
||||
- Fleet operations: [`docs/guides/fleet-local-canary.md`](./fleet-local-canary.md)
|
||||
- Ownership SSOT: [`packages/mosaic/framework/framework-manifest.txt`](../../packages/mosaic/framework/framework-manifest.txt)
|
||||
@@ -0,0 +1,621 @@
|
||||
# Mosaic Stack — User Guide
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Getting Started](#getting-started)
|
||||
2. [Chat Interface](#chat-interface)
|
||||
3. [Projects](#projects)
|
||||
4. [Tasks](#tasks)
|
||||
5. [Settings](#settings)
|
||||
6. [CLI Usage](#cli-usage)
|
||||
7. [Sub-package Commands](#sub-package-commands)
|
||||
8. [Telemetry](#telemetry)
|
||||
9. [Local Fleet Canary](./fleet-local-canary.md)
|
||||
|
||||
---
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Prerequisites
|
||||
|
||||
Mosaic Stack requires a running gateway. Your administrator provides the URL
|
||||
(default: `http://localhost:14242`) and creates your account.
|
||||
|
||||
### Logging In (Web)
|
||||
|
||||
1. Navigate to the Mosaic web app (default: `http://localhost:3000`).
|
||||
2. You are redirected to `/login` automatically.
|
||||
3. Enter your email and password, then click **Sign in**.
|
||||
4. On success you land on the **Chat** page.
|
||||
|
||||
### Registering an Account
|
||||
|
||||
If self-registration is enabled:
|
||||
|
||||
1. Go to `/register`.
|
||||
2. Enter your name, email, and password.
|
||||
3. Submit. You are signed in and redirected to Chat.
|
||||
|
||||
---
|
||||
|
||||
## Chat Interface
|
||||
|
||||
### Sending a Message
|
||||
|
||||
1. Type your message in the input bar at the bottom of the Chat page.
|
||||
2. Press **Enter** to send.
|
||||
3. The assistant response streams in real time. A spinner indicates the agent is
|
||||
processing.
|
||||
|
||||
### Streaming Responses
|
||||
|
||||
Responses appear token by token as the model generates them. You can read the
|
||||
response while it is still being produced. The streaming indicator clears when
|
||||
the response is complete.
|
||||
|
||||
### Conversation Management
|
||||
|
||||
- **New conversation**: Navigate to `/chat` or click **New Chat** in the sidebar.
|
||||
A new conversation ID is created automatically on your first message.
|
||||
- **Resume a conversation**: Conversations are stored server-side. Refresh the
|
||||
page or navigate away and back to continue where you left off. The current
|
||||
conversation ID is shown in the URL.
|
||||
- **Conversation list**: The sidebar shows recent conversations. Click any entry
|
||||
to switch.
|
||||
|
||||
### Model and Provider
|
||||
|
||||
The current model and provider are displayed in the chat header. To change them,
|
||||
use the Settings page (see [Provider Settings](#providers)) or the CLI
|
||||
`/model` and `/provider` commands.
|
||||
|
||||
---
|
||||
|
||||
## Projects
|
||||
|
||||
Projects group related missions and tasks. Navigate to **Projects** in the
|
||||
sidebar.
|
||||
|
||||
### Creating a Project
|
||||
|
||||
1. Go to `/projects`.
|
||||
2. Click **New Project**.
|
||||
3. Enter a name and optional description.
|
||||
4. Select a status: `active`, `paused`, `completed`, or `archived`.
|
||||
5. Save. The project appears in the list.
|
||||
|
||||
### Viewing a Project
|
||||
|
||||
Click a project card to open its detail view at `/projects/<id>`. From here you
|
||||
can see the project's missions, tasks, and metadata.
|
||||
|
||||
### Managing Tasks within a Project
|
||||
|
||||
Tasks are linked to projects and optionally to missions. See [Tasks](#tasks) for
|
||||
full details. On the project detail page, the task list is filtered to the
|
||||
selected project.
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
Navigate to **Tasks** in the sidebar to see all tasks across all projects.
|
||||
|
||||
### Task Statuses
|
||||
|
||||
| Status | Meaning |
|
||||
| ------------- | ------------------------ |
|
||||
| `not-started` | Not yet started |
|
||||
| `in-progress` | Actively being worked on |
|
||||
| `blocked` | Waiting on something |
|
||||
| `done` | Completed |
|
||||
| `cancelled` | No longer needed |
|
||||
|
||||
### Creating a Task
|
||||
|
||||
1. Go to `/tasks`.
|
||||
2. Click **New Task**.
|
||||
3. Enter a title, optional description, and link to a project or mission.
|
||||
4. Set the status and priority.
|
||||
5. Save.
|
||||
|
||||
### Updating a Task
|
||||
|
||||
Click a task to open its detail panel. Edit the fields inline and save.
|
||||
|
||||
---
|
||||
|
||||
## Settings
|
||||
|
||||
Navigate to **Settings** in the sidebar (or `/settings`) to manage your profile,
|
||||
appearance, and providers.
|
||||
|
||||
### Profile Tab
|
||||
|
||||
- **Name**: Display name shown in the UI.
|
||||
- **Email**: Read-only; contact your administrator to change email.
|
||||
- Changes save automatically when you click **Save Profile**.
|
||||
|
||||
### Appearance Tab
|
||||
|
||||
- **Theme**: Choose `light`, `dark`, or `system`.
|
||||
- The theme preference is saved to your account and applies on all devices.
|
||||
|
||||
### Notifications Tab
|
||||
|
||||
Configure notification preferences (future feature; placeholder in the current
|
||||
release).
|
||||
|
||||
### Providers Tab
|
||||
|
||||
View all configured LLM providers and their models.
|
||||
|
||||
- **Test Connection**: Click **Test** next to a provider to check reachability.
|
||||
The result shows latency and discovered models.
|
||||
- Provider configuration is managed by your administrator via environment
|
||||
variables. See the [Admin Guide](./admin-guide.md) for setup.
|
||||
|
||||
---
|
||||
|
||||
## CLI Usage
|
||||
|
||||
The `mosaic` CLI provides a terminal interface to the same gateway API.
|
||||
|
||||
### Installation
|
||||
|
||||
Install via the Mosaic installer:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://mosaicstack.dev/install.sh | bash
|
||||
```
|
||||
|
||||
Or use the direct URL:
|
||||
|
||||
```bash
|
||||
bash <(curl -fsSL https://git.mosaicstack.dev/mosaicstack/stack/raw/branch/main/tools/install.sh)
|
||||
```
|
||||
|
||||
The installer places the `mosaic` binary at `~/.npm-global/bin/mosaic`.
|
||||
|
||||
Install lanes:
|
||||
|
||||
| Lane | Command | Source |
|
||||
| ------------------------ | ------------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| Stable | `bash tools/install.sh` | npm `@mosaicstack/mosaic@latest` + `main` |
|
||||
| Prerelease integration | `bash tools/install.sh --next` | Fast npm `@mosaicstack/mosaic@next` + `@mosaicstack/gateway@next`; source fallback at `next` |
|
||||
| Contributor/source build | `bash tools/install.sh --dev --ref X` | Build-from-source at the requested ref |
|
||||
|
||||
`--next` is fast-by-default from the Gitea npm `next` dist-tag and falls back to a source build at the permanent `next` branch if the dist-tag is missing or unreachable. Explicit `--ref` or `MOSAIC_REF` still wins and uses the source path.
|
||||
Flags for non-interactive use:
|
||||
|
||||
```bash
|
||||
--yes # Accept all defaults
|
||||
--no-auto-launch # Skip auto-launch of wizard after install
|
||||
```
|
||||
|
||||
Unrecognized flags or positional arguments fail before installation starts and print the supported-option usage.
|
||||
|
||||
Or if installed globally:
|
||||
|
||||
```bash
|
||||
mosaic --help
|
||||
```
|
||||
|
||||
### First-Run Wizard
|
||||
|
||||
After install the wizard launches automatically. You can re-run it at any time:
|
||||
|
||||
```bash
|
||||
mosaic wizard
|
||||
```
|
||||
|
||||
The wizard guides you through:
|
||||
|
||||
1. Gateway discovery or installation (`mosaic gateway install`)
|
||||
2. Authentication (`mosaic gateway login`)
|
||||
3. Post-install health check (`mosaic gateway verify`)
|
||||
|
||||
### Gateway Login and Token Recovery
|
||||
|
||||
```bash
|
||||
# Authenticate with a gateway and save a session token
|
||||
mosaic gateway login
|
||||
|
||||
# Verify the gateway is reachable and responding
|
||||
mosaic gateway verify
|
||||
|
||||
# Rotate your current API token
|
||||
mosaic gateway config rotate-token
|
||||
|
||||
# Recover a token via BetterAuth cookie (for accounts with no token)
|
||||
mosaic gateway config recover-token
|
||||
```
|
||||
|
||||
If you have an existing gateway account but lost your token (common after a
|
||||
reinstall), use `mosaic gateway config recover-token` to retrieve a new one
|
||||
without recreating your account.
|
||||
|
||||
### Configuration
|
||||
|
||||
```bash
|
||||
# Print full config as JSON
|
||||
mosaic config show
|
||||
|
||||
# Read a specific key
|
||||
mosaic config get gateway.url
|
||||
|
||||
# Write a key
|
||||
mosaic config set gateway.url http://localhost:14242
|
||||
|
||||
# Open config in $EDITOR
|
||||
mosaic config edit
|
||||
|
||||
# Print config file path
|
||||
mosaic config path
|
||||
```
|
||||
|
||||
### Signing In (Legacy)
|
||||
|
||||
```bash
|
||||
mosaic login --gateway http://localhost:14242 --email you@example.com
|
||||
```
|
||||
|
||||
You are prompted for a password if `--password` is not supplied. The session
|
||||
cookie is saved locally and reused on subsequent commands.
|
||||
|
||||
### Launching the TUI
|
||||
|
||||
```bash
|
||||
mosaic tui
|
||||
```
|
||||
|
||||
Options:
|
||||
|
||||
| Flag | Default | Description |
|
||||
| ----------------------- | ------------------------ | ---------------------------------- |
|
||||
| `--gateway <url>` | `http://localhost:14242` | Gateway URL |
|
||||
| `--conversation <id>` | — | Resume a specific conversation |
|
||||
| `--model <modelId>` | server default | Model to use (e.g. `llama3.2`) |
|
||||
| `--provider <provider>` | server default | Provider (e.g. `ollama`, `openai`) |
|
||||
|
||||
If no valid session exists you are prompted to sign in before the TUI launches.
|
||||
|
||||
### TUI Slash Commands
|
||||
|
||||
Inside the TUI, type a `/` command and press Enter:
|
||||
|
||||
| Command | Description |
|
||||
| ---------------------- | ------------------------------ |
|
||||
| `/model <modelId>` | Switch to a different model |
|
||||
| `/provider <provider>` | Switch to a different provider |
|
||||
| `/models` | List available models |
|
||||
| `/exit` or `/quit` | Exit the TUI |
|
||||
|
||||
### Session Management
|
||||
|
||||
```bash
|
||||
# List saved sessions
|
||||
mosaic sessions list
|
||||
|
||||
# Resume a session
|
||||
mosaic sessions resume <sessionId>
|
||||
|
||||
# Destroy a session
|
||||
mosaic sessions destroy <sessionId>
|
||||
```
|
||||
|
||||
### Other Commands
|
||||
|
||||
```bash
|
||||
# Run the Mosaic installation wizard
|
||||
mosaic wizard
|
||||
|
||||
# PRD wizard (generate product requirement documents)
|
||||
mosaic prdy
|
||||
|
||||
# Quality rails scaffolder
|
||||
mosaic quality-rails
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Claude Code Skill Registration
|
||||
|
||||
Mosaic stores canonical skills under `~/.config/mosaic/skills/`. Claude Code scans
|
||||
`~/.claude/skills/`, so Mosaic maintains one symlink per skill between those
|
||||
directories.
|
||||
|
||||
```bash
|
||||
mosaic skill list
|
||||
mosaic skill register <name>
|
||||
mosaic skill unregister <name>
|
||||
```
|
||||
|
||||
- `register` is idempotent and repairs a dangling Mosaic-owned link. Names use
|
||||
the safe grammar `[A-Za-z0-9][A-Za-z0-9._-]*`; files, directories, foreign
|
||||
symlinks, path traversal, absolute paths, and names beginning with `-` are
|
||||
refused.
|
||||
- `unregister` is idempotent when no entry exists. It removes only symlinks that
|
||||
point inside `~/.config/mosaic/skills/`; foreign entries are never removed.
|
||||
- `list` reports `registered`, `unregistered`, `dangling`, `foreign`,
|
||||
`foreign-dangling`, or `misdirected` for each canonical or Claude entry.
|
||||
|
||||
Install, wizard finalization, and `mosaic update` framework re-seeding reconcile
|
||||
every canonical skill automatically. A skill directory added after initial
|
||||
setup therefore receives its Claude bridge without a per-skill code change or
|
||||
manual `ln -s`. If Claude Code is already running, use `/reload-skills` or start
|
||||
a new session after registration so its in-process skill registry rescans.
|
||||
|
||||
This command group is Claude-only in M1. Pi can consume Mosaic's canonical skill
|
||||
root through its Mosaic launcher configuration and does not need this Claude
|
||||
bridge. Codex has a separate link path managed by the legacy full skill-sync
|
||||
script; equivalent lifecycle management remains follow-up scope and is not
|
||||
changed here.
|
||||
|
||||
## Sub-package Commands
|
||||
|
||||
Each Mosaic sub-package exposes its full API surface through the `mosaic` CLI.
|
||||
All sub-package commands accept `--help` for usage details.
|
||||
|
||||
### `mosaic auth` — User & Authentication Management
|
||||
|
||||
Manage gateway users, SSO providers, and active sessions.
|
||||
|
||||
```bash
|
||||
# List all users
|
||||
mosaic auth users list
|
||||
|
||||
# Create a new user
|
||||
mosaic auth users create --email alice@example.com --name "Alice"
|
||||
|
||||
# Delete a user
|
||||
mosaic auth users delete <userId>
|
||||
|
||||
# List configured SSO providers
|
||||
mosaic auth sso
|
||||
|
||||
# List active sessions
|
||||
mosaic auth sessions list
|
||||
|
||||
# Revoke a session
|
||||
mosaic auth sessions revoke <sessionId>
|
||||
```
|
||||
|
||||
### `mosaic brain` — Projects, Missions, Tasks, Conversations
|
||||
|
||||
Browse and manage the brain data layer (PostgreSQL-backed project/mission/task
|
||||
store).
|
||||
|
||||
```bash
|
||||
# List all projects
|
||||
mosaic brain projects
|
||||
|
||||
# List missions for a project
|
||||
mosaic brain missions --project <projectId>
|
||||
|
||||
# List tasks
|
||||
mosaic brain tasks --status in-progress
|
||||
|
||||
# Browse conversations
|
||||
mosaic brain conversations
|
||||
mosaic brain conversations --project <projectId>
|
||||
```
|
||||
|
||||
### `mosaic config` — CLI Configuration
|
||||
|
||||
Read and write the `mosaic` CLI configuration file.
|
||||
|
||||
```bash
|
||||
# Show full config
|
||||
mosaic config show
|
||||
|
||||
# Get a value
|
||||
mosaic config get gateway.url
|
||||
|
||||
# Set a value
|
||||
mosaic config set theme dark
|
||||
|
||||
# Open in editor
|
||||
mosaic config edit
|
||||
|
||||
# Print file path
|
||||
mosaic config path
|
||||
```
|
||||
|
||||
### `mosaic forge` — AI Pipeline Management
|
||||
|
||||
Interact with the Forge multi-stage AI delivery pipeline (intake → board review
|
||||
→ planning → coding → review → deploy).
|
||||
|
||||
```bash
|
||||
# Start a new forge run for a brief
|
||||
mosaic forge run --brief "Add dark mode toggle to settings"
|
||||
|
||||
# Check status of a running pipeline
|
||||
mosaic forge status
|
||||
mosaic forge status --run <runId>
|
||||
|
||||
# Resume a paused or interrupted run
|
||||
mosaic forge resume --run <runId>
|
||||
|
||||
# List available personas (board review evaluators)
|
||||
mosaic forge personas
|
||||
```
|
||||
|
||||
### `mosaic gateway` — Gateway Lifecycle
|
||||
|
||||
Install, authenticate with, and verify the Mosaic gateway service.
|
||||
|
||||
```bash
|
||||
# Install gateway (guided)
|
||||
mosaic gateway install
|
||||
|
||||
# Verify gateway health post-install
|
||||
mosaic gateway verify
|
||||
|
||||
# Log in and save token
|
||||
mosaic gateway login
|
||||
|
||||
# Rotate API token
|
||||
mosaic gateway config rotate-token
|
||||
|
||||
# Recover token via BetterAuth cookie (lost-token recovery)
|
||||
mosaic gateway config recover-token
|
||||
```
|
||||
|
||||
### `mosaic log` — Structured Log Access
|
||||
|
||||
Query and stream structured logs from the gateway.
|
||||
|
||||
```bash
|
||||
# Stream live logs
|
||||
mosaic log tail
|
||||
mosaic log tail --level warn
|
||||
|
||||
# Search logs
|
||||
mosaic log search "database connection"
|
||||
mosaic log search --since 1h "error"
|
||||
|
||||
# Export logs to file
|
||||
mosaic log export --output logs.json
|
||||
mosaic log export --since 24h --level error --output errors.json
|
||||
|
||||
# Get/set log level
|
||||
mosaic log level
|
||||
mosaic log level debug
|
||||
```
|
||||
|
||||
### `mosaic macp` — MACP Protocol
|
||||
|
||||
Interact with the MACP credential resolution, gate runner, and event bus.
|
||||
|
||||
```bash
|
||||
# List MACP tasks
|
||||
mosaic macp tasks
|
||||
mosaic macp tasks --status pending
|
||||
|
||||
# Submit a new MACP task
|
||||
mosaic macp submit --type credential-resolve --payload '{"key":"OPENAI_API_KEY"}'
|
||||
|
||||
# Run a gate check
|
||||
mosaic macp gate --gate quality-check
|
||||
|
||||
# Stream MACP events
|
||||
mosaic macp events
|
||||
mosaic macp events --filter credential
|
||||
```
|
||||
|
||||
### `mosaic memory` — Agent Memory
|
||||
|
||||
Query and inspect the agent memory layer.
|
||||
|
||||
```bash
|
||||
# Semantic search over memory
|
||||
mosaic memory search "previous decisions about auth"
|
||||
|
||||
# Show memory statistics
|
||||
mosaic memory stats
|
||||
|
||||
# Generate memory insights report
|
||||
mosaic memory insights
|
||||
|
||||
# View stored preferences
|
||||
mosaic memory preferences
|
||||
mosaic memory preferences --set editor=neovim
|
||||
```
|
||||
|
||||
### `mosaic queue` — Task Queue (Valkey)
|
||||
|
||||
Manage the Valkey-backed task queue.
|
||||
|
||||
```bash
|
||||
# List all queues
|
||||
mosaic queue list
|
||||
|
||||
# Show queue statistics
|
||||
mosaic queue stats
|
||||
mosaic queue stats --queue agent-tasks
|
||||
|
||||
# Pause a queue
|
||||
mosaic queue pause agent-tasks
|
||||
|
||||
# Resume a paused queue
|
||||
mosaic queue resume agent-tasks
|
||||
|
||||
# List jobs in a queue
|
||||
mosaic queue jobs agent-tasks
|
||||
mosaic queue jobs agent-tasks --status failed
|
||||
|
||||
# Drain (empty) a queue
|
||||
mosaic queue drain agent-tasks
|
||||
```
|
||||
|
||||
### `mosaic storage` — Object Storage
|
||||
|
||||
Manage object storage tiers and data migrations.
|
||||
|
||||
```bash
|
||||
# Show storage status and usage
|
||||
mosaic storage status
|
||||
|
||||
# List available storage tiers
|
||||
mosaic storage tier
|
||||
|
||||
# Export data from storage
|
||||
mosaic storage export --bucket agent-artifacts --output ./artifacts.tar.gz
|
||||
|
||||
# Import data into storage
|
||||
mosaic storage import --bucket agent-artifacts --input ./artifacts.tar.gz
|
||||
|
||||
# Schema migration is unavailable in this release. The current storage wrapper shells
|
||||
# directly to `pnpm --filter @mosaicstack/db db:migrate`; it is legacy N-1,
|
||||
# uncertified, and MUST NOT be invoked pending KBN-101-02/-03/-06/-08 activation.
|
||||
# Future schema migration is non-operative: external bootstrap → TLS/roles → runner
|
||||
# --run → runner --verify → readiness.
|
||||
|
||||
# Tier copy uses only the separately held secure migrate-tier route. Never use a legacy
|
||||
# --from/--to storage-migrate command or pass a credential on argv.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Telemetry
|
||||
|
||||
Mosaic includes an OpenTelemetry-based telemetry system. Local telemetry
|
||||
(traces, metrics sent to Jaeger) is always available. Remote telemetry upload
|
||||
requires explicit opt-in.
|
||||
|
||||
### Local Telemetry
|
||||
|
||||
```bash
|
||||
# Show local OTEL collector / Jaeger status
|
||||
mosaic telemetry local status
|
||||
|
||||
# Tail live OTEL spans
|
||||
mosaic telemetry local tail
|
||||
|
||||
# Open Jaeger UI URL
|
||||
mosaic telemetry local jaeger
|
||||
```
|
||||
|
||||
### Remote Telemetry
|
||||
|
||||
Remote upload is a no-op (dry-run) until you opt in. Your consent state is
|
||||
persisted in the config file.
|
||||
|
||||
```bash
|
||||
# Show current consent state
|
||||
mosaic telemetry status
|
||||
|
||||
# Opt in to remote telemetry
|
||||
mosaic telemetry opt-in
|
||||
|
||||
# Opt out (data stays local)
|
||||
mosaic telemetry opt-out
|
||||
|
||||
# Test telemetry pipeline without uploading
|
||||
mosaic telemetry test
|
||||
|
||||
# Upload telemetry (requires opt-in; dry-run otherwise)
|
||||
mosaic telemetry upload
|
||||
```
|
||||
@@ -0,0 +1,101 @@
|
||||
# Mission Control Plane — Feature Board
|
||||
|
||||
> Discussion board for the combined PRD / mission / Kanban workflow.
|
||||
> Use this to decide scope before implementation.
|
||||
|
||||
## Board Legend
|
||||
|
||||
- **Must-have** — required for the first usable version
|
||||
- **Should-have** — strongly preferred, but can ship after the core path
|
||||
- **Could-have** — valuable later if time permits
|
||||
- **Won't-have** — explicitly deferred
|
||||
|
||||
---
|
||||
|
||||
## Feature Board
|
||||
|
||||
| Feature Card | Need | Priority | Decision / Notes |
|
||||
| ------------------------------ | ------------------------------------------------------------- | ----------- | --------------------------------------------------------------------------- |
|
||||
| Canonical mission manifest | One durable root object for goal, PRD, board, session | Must-have | Mission manifest becomes the anchor for all downstream state |
|
||||
| PRD generator integration | PRD should be generated from a feature idea and saved in docs | Must-have | Use Mosaic PRDy format and keep the file human-reviewable |
|
||||
| Board atomization | Break PRD into assignable tasks with dependencies | Must-have | Each user story should map to one or more tasks |
|
||||
| Short-cycle detector | Detect compaction churn and repeated tool loops | Must-have | Coordinator should track churn score per session |
|
||||
| Handoff packet | Preserve actionable context across rotations | Must-have | Use a compact structured summary, not a raw transcript |
|
||||
| Auto-resume workers | Let new sessions read mission + board on start | Should-have | Makes overnight autonomy realistic |
|
||||
| Mission status view | Show current phase, blockers, and active session | Should-have | Expose through CLI first, dashboard later |
|
||||
| Worktree root convention | Keep worktrees off `/tmp` and on the larger persistent drive | Should-have | Prefer `/src/<repo>-worktrees` for repo worktrees and long-lived agent work |
|
||||
| Review gate | Prevent autonomous work from shipping unreviewed | Should-have | Use reviewer tasks before mission close |
|
||||
| Rotation policy config | Configure thresholds per mission/profile | Could-have | Keep v1 simple, add tuning later |
|
||||
| Goal decomposition suggestions | Suggest sub-goals from the PRD | Could-have | Good for planning, not necessary for core path |
|
||||
| Cross-channel continuity | Continue a mission across CLI/gateway/remote channels | Could-have | Important later, not required for MVP |
|
||||
| Automatic board sync | Mirror git docs into DB and back | Could-have | Nice-to-have after the file-first flow stabilizes |
|
||||
| Fully autonomous closeout | Let mission finish without human intervention | Won't-have | Keep an operator-visible review step |
|
||||
|
||||
---
|
||||
|
||||
## Needs Discussion
|
||||
|
||||
### 1) Canonical source of truth
|
||||
|
||||
**Question:** Should the PRD, mission manifest, and board all live in git, or should one be the database source of truth?
|
||||
|
||||
**Proposed answer:** Keep the human-readable artifacts in git and sync the mission runtime state to the database.
|
||||
|
||||
### 2) Scope of automation
|
||||
|
||||
**Question:** Should the first version auto-create the board from the PRD, or require a human/orchestrator to approve the split?
|
||||
|
||||
**Proposed answer:** Auto-create a draft board, then let the orchestrator approve or adjust it.
|
||||
|
||||
### 3) Rotation triggers
|
||||
|
||||
**Question:** What should trigger a forced session rotation?
|
||||
|
||||
**Candidate signals:**
|
||||
|
||||
- repeated compaction
|
||||
- repeated prompts for permission
|
||||
- identical tool loops
|
||||
- no new file/task state after several turns
|
||||
- task blocked on a missing prerequisite
|
||||
|
||||
**Proposed answer:** Use a weighted churn score with a small hard cap on repeated compactions.
|
||||
|
||||
### 4) Handoff format
|
||||
|
||||
**Question:** What should the next session receive?
|
||||
|
||||
**Proposed answer:**
|
||||
|
||||
- Mission ID
|
||||
- PRD path
|
||||
- Active board task
|
||||
- Completed work
|
||||
- Blockers
|
||||
- Next 3 actions
|
||||
- Non-negotiable constraints
|
||||
|
||||
### 5) Operator control
|
||||
|
||||
**Question:** Should the operator be able to force a rotation or pause the mission?
|
||||
|
||||
**Proposed answer:** Yes. Human override should win.
|
||||
|
||||
---
|
||||
|
||||
## Draft Decisions
|
||||
|
||||
1. File-first artifacts, DB-backed runtime state.
|
||||
2. PRD-first planning, board-second execution.
|
||||
3. Auto-rotation on churn, but human override remains available.
|
||||
4. Structured handoff packets required on every rotation.
|
||||
5. Mission close requires a reviewer task.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
- What exact data fields belong in the mission manifest?
|
||||
- Should rotation thresholds vary by agent profile?
|
||||
- What is the minimum viable status surface for v1?
|
||||
- Should the board support milestones in addition to tasks?
|
||||
@@ -0,0 +1,95 @@
|
||||
# Mission Manifest — Mosaic Mission Control Plane
|
||||
|
||||
> Persistent document tracking scope, status, and handoff history for the combined PRD / mission / Kanban workflow.
|
||||
|
||||
## Mission
|
||||
|
||||
**ID:** mission-control-plane-20260506
|
||||
|
||||
**Statement:** Combine Mosaic PRDy, coord, and Kanban into one durable workflow so an agent can move from feature idea to PRD to mission to task board and keep working across session rotation, compaction, and restarts with minimal context loss.
|
||||
|
||||
**Phase:** planning — MC-01 complete, MC-02 next
|
||||
|
||||
**Current Milestone:** MC-02
|
||||
|
||||
**Progress:** 1 / 6 milestones
|
||||
|
||||
**Status:** active
|
||||
|
||||
**Last Updated:** 2026-05-06
|
||||
|
||||
**Parent Mission:** None — new mission
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
This mission exists because overnight autonomy breaks when the working session short-cycles. The system needs durable artifacts and a mechanical coordinator that can:
|
||||
|
||||
1. keep a canonical PRD,
|
||||
2. atomize the PRD into board tasks,
|
||||
3. track mission state separately from the chat session,
|
||||
4. detect churn or compaction pressure,
|
||||
5. rotate to a fresh session, and
|
||||
6. re-enter from a structured handoff.
|
||||
|
||||
Operational convention: repo worktrees and long-lived working directories should use `/src/<repo>-worktrees` instead of `/tmp`.
|
||||
|
||||
Design references:
|
||||
|
||||
- `docs/mission-control/PRD.md` — product requirements
|
||||
- `docs/mission-control/BOARD.md` — feature discussion board
|
||||
- `docs/mission-control/TASKS.md` — atomized execution plan
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] AC-1: A feature idea can be converted into a PRD, mission, and task board.
|
||||
- [ ] AC-2: The coordinator can load a mission and its board from durable storage.
|
||||
- [ ] AC-3: The coordinator can detect short-cycling and rotate sessions automatically.
|
||||
- [ ] AC-4: A rotated session can resume from a handoff packet without manual re-prompting.
|
||||
- [ ] AC-5: The board remains traceable back to the PRD user stories.
|
||||
- [ ] AC-6: Operators can inspect mission state, task state, and latest handoff from one place.
|
||||
- [ ] AC-7: The system can run overnight without losing the mission goal.
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
| # | ID | Name | Status | Branch | Started | Completed |
|
||||
| --- | ----- | ---------------------------------------- | ----------- | ----------------------- | ---------- | --------- |
|
||||
| 1 | MC-01 | PRD + mission schema foundation | in-progress | docs/mission-control-\* | 2026-05-06 | — |
|
||||
| 2 | MC-02 | Mission runtime model | not-started | — | — | — |
|
||||
| 3 | MC-03 | Board atomization and task linkage | not-started | — | — | — |
|
||||
| 4 | MC-04 | Short-cycle detector and rotation engine | not-started | — | — | — |
|
||||
| 5 | MC-05 | Handoff generation and re-entry | not-started | — | — | — |
|
||||
| 6 | MC-06 | Operator surface and E2E validation | not-started | — | — | — |
|
||||
|
||||
---
|
||||
|
||||
## Budget
|
||||
|
||||
| Milestone | Est. tokens | Parallelizable? |
|
||||
| --------- | ----------- | ------------------ |
|
||||
| MC-01 | 16K | No |
|
||||
| MC-02 | 20K | No |
|
||||
| MC-03 | 24K | Mostly after MC-01 |
|
||||
| MC-04 | 20K | After MC-02 |
|
||||
| MC-05 | 18K | After MC-04 |
|
||||
| MC-06 | 26K | After MC-04/05 |
|
||||
| **Total** | **~124K** | |
|
||||
|
||||
---
|
||||
|
||||
## Session History
|
||||
|
||||
| Session | Date | Runtime | Outcome |
|
||||
| ------- | ---------- | ------- | ------------------------------------------------------------------------ |
|
||||
| S1 | 2026-05-06 | hermes | PRD, board, task plan, mission manifest, and worktree convention drafted |
|
||||
|
||||
---
|
||||
|
||||
## Next Step
|
||||
|
||||
Kick off MC-02: implement the durable mission runtime model and wire the mission state into the coordinator.
|
||||
@@ -0,0 +1,205 @@
|
||||
# PRD: Mosaic Mission Control Plane
|
||||
|
||||
## Metadata
|
||||
|
||||
- **Owner:** Jason Woltje
|
||||
- **Date:** 2026-05-06
|
||||
- **Status:** draft
|
||||
- **Framework:** Mosaic PRDy + coord + Kanban
|
||||
- **Target Repo:** `git.mosaicstack.dev/mosaic/mosaic-stack`
|
||||
- **Primary Modules:** `packages/prdy`, `packages/coord`, `packages/queue`, `apps/gateway`, `packages/brain`, `packages/cli`
|
||||
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Mosaic already has the ingredients for durable agent work: PRD generation (`prdy`), mission coordination (`coord`), and task execution boards (`Kanban` / `TASKS.md`). Today those systems can still drift apart:
|
||||
|
||||
- A PRD can exist without a mission record.
|
||||
- A mission can exist without a machine-readable execution board.
|
||||
- Agents can short-cycle or compact repeatedly without a durable handoff.
|
||||
- The next session may know the goal, but not the exact next step.
|
||||
|
||||
The result is brittle overnight autonomy: work continues only as long as a single session remains healthy.
|
||||
|
||||
This feature unifies those layers into one durable workflow so a mission can survive session rotation, compaction, and restarts with minimal state loss.
|
||||
|
||||
---
|
||||
|
||||
## Goals
|
||||
|
||||
1. Create one canonical pipeline from idea → PRD → mission → board → execution.
|
||||
2. Let `prdy` generate a PRD that is immediately usable as a mission input.
|
||||
3. Let `coord` own mission state, handoffs, and session rotation.
|
||||
4. Let the board hold atomized tasks with dependencies and assignees.
|
||||
5. Let agents read the mission and board to learn the next action without extra prompting.
|
||||
6. Detect short-cycling and rotate sessions before quality degrades.
|
||||
7. Preserve useful context across handoffs with a structured summary packet.
|
||||
8. Give operators a single place to see mission status, task state, and the current session.
|
||||
|
||||
---
|
||||
|
||||
## Non-Goals
|
||||
|
||||
1. Replacing the Mosaic agent runtime or gateway architecture.
|
||||
2. Rewriting `prdy` or `coord` from scratch.
|
||||
3. Turning the board into a general project-management system.
|
||||
4. Building a full Gantt/charting product.
|
||||
5. Removing human review or approval gates.
|
||||
6. Allowing agents to create arbitrary mission state without schema.
|
||||
|
||||
---
|
||||
|
||||
## User Stories
|
||||
|
||||
### US-001: Create a mission from a feature idea
|
||||
|
||||
**Description:** As an orchestrator, I want to turn a feature idea into a PRD and mission so that agents can work from a durable spec instead of a chat transcript.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
|
||||
- [ ] `prdy` can emit a PRD with goals, non-goals, and requirements.
|
||||
- [ ] The PRD is linked to a mission ID.
|
||||
- [ ] The mission manifest references the PRD path.
|
||||
- [ ] The mission is readable by downstream agent sessions.
|
||||
|
||||
### US-002: Atomize work into a board
|
||||
|
||||
**Description:** As an orchestrator, I want to split a PRD into board tasks so that work can be assigned to specialists.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
|
||||
- [ ] Each user story can become one or more tasks.
|
||||
- [ ] Tasks have assignees, dependencies, and estimates.
|
||||
- [ ] Tasks are machine-readable and durable.
|
||||
- [ ] The board can be regenerated from the PRD without ambiguity.
|
||||
|
||||
### US-003: Rotate sessions without losing the mission
|
||||
|
||||
**Description:** As a coordinator, I want to restart or rotate a session when it short-cycles so that the mission continues with minimal loss.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
|
||||
- [ ] The coordinator detects compaction pressure or repeated loops.
|
||||
- [ ] The coordinator writes a handoff summary before rotation.
|
||||
- [ ] A new session can resume from the handoff packet.
|
||||
- [ ] The mission state remains intact across the rotation.
|
||||
|
||||
### US-004: Let workers read the next step automatically
|
||||
|
||||
**Description:** As a worker agent, I want to read the mission and board at startup so I can do the next useful thing without waiting for a human prompt.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
|
||||
- [ ] Startup loads the active mission manifest.
|
||||
- [ ] Startup loads the current board/task row.
|
||||
- [ ] Startup exposes the next action clearly in the prompt.
|
||||
- [ ] The agent can continue after compaction using the same mission context.
|
||||
|
||||
### US-005: Observe mission health from one place
|
||||
|
||||
**Description:** As an operator, I want a single view of mission health so that I can see progress, blocked tasks, and session churn.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
|
||||
- [ ] Mission state shows current phase and progress.
|
||||
- [ ] Board state shows task status by assignee.
|
||||
- [ ] Short-cycle/rotation events are visible.
|
||||
- [ ] Handoffs are inspectable.
|
||||
|
||||
---
|
||||
|
||||
## Functional Requirements
|
||||
|
||||
FR-1. The system must represent a mission as a durable object with an ID, goal, current phase, PRD path, board path, and active session ID.
|
||||
|
||||
FR-2. The system must represent a PRD as a markdown document with goals, user stories, functional requirements, non-goals, technical considerations, and success metrics.
|
||||
|
||||
FR-3. The system must represent execution work as a board of atomized tasks with status, assignee, dependency, and estimate fields.
|
||||
|
||||
FR-4. The coordinator must be able to derive a task board from a PRD.
|
||||
|
||||
FR-5. The coordinator must be able to write a handoff packet that includes goal, current state, completed work, blocked work, next steps, and constraints.
|
||||
|
||||
FR-6. The coordinator must detect short-cycling signals such as repeated compactions, repeated tool loops, repeated approval prompts, or no progress across several turns.
|
||||
|
||||
FR-7. The coordinator must rotate the session when the short-cycle threshold is exceeded.
|
||||
|
||||
FR-8. The coordinator must preserve mission continuity across session rotation.
|
||||
|
||||
FR-9. The worker session must read the mission state and board state at startup.
|
||||
|
||||
FR-10. The worker session must be able to resume from the last handoff summary without the operator rewriting the goal manually.
|
||||
|
||||
FR-11. The operator must be able to inspect the mission state, PRD, board, and latest handoff from one place.
|
||||
|
||||
FR-12. The mission system must keep a traceable link between PRD requirements and board tasks.
|
||||
|
||||
FR-13. The system must not allow a task to become active without a valid mission context.
|
||||
|
||||
FR-14. The system must keep durable history for rotation and handoff events.
|
||||
|
||||
---
|
||||
|
||||
## Board Discussion: Features and Needs
|
||||
|
||||
This is the feature discussion board that should drive the mission design.
|
||||
|
||||
| Card | Need | Why it matters | Proposed decision |
|
||||
| ------------------------ | -------------------------------------------- | -------------------------------------------- | ------------------------------------------------------------ |
|
||||
| Canonical mission record | One source of truth for goal/state | Prevents drift between chat, docs, and queue | Make mission manifest the durable root object |
|
||||
| PRD → board derivation | Break feature ideas into executable work | Lets the plan be assigned and tracked | Keep PRD as the spec, generate board tasks from user stories |
|
||||
| Session watchdog | Detect churn/short-cycling | Keeps overnight runs productive | Add short-cycle scoring and forced rotation |
|
||||
| Structured handoff | Preserve context across session changes | Minimizes restart loss | Use a compact JSON/MD handoff packet |
|
||||
| Worker auto-read | Let agents resume without human re-prompting | Reduces operator overhead | Load mission + board on session start |
|
||||
| Status surface | Show progress and blockers clearly | Operators need confidence | Expose mission state via CLI and dashboard |
|
||||
| Review gate | Keep quality high on autonomous work | Prevents silent regressions | Require review tasks before close |
|
||||
| Recoverability | Resume after failure or restart | Mission should outlive a process | Persist session and handoff history |
|
||||
|
||||
---
|
||||
|
||||
## Design Considerations
|
||||
|
||||
1. The PRD should stay human-readable markdown, because the board and mission references need to be reviewable in git.
|
||||
2. The board should be machine-readable enough for automation but still readable by humans.
|
||||
3. The mission manifest should point to the PRD and board, not duplicate them.
|
||||
4. Handoff packets should be compact and structured so they can be injected into a new session with minimal token cost.
|
||||
5. The coordinator should prefer rotation over forced context growth once the session is near the compaction threshold.
|
||||
6. Existing Mosaic commands should be extended, not replaced, wherever possible.
|
||||
7. The same mission should be resumable across CLI, gateway, and remote channels.
|
||||
|
||||
---
|
||||
|
||||
## Technical Considerations
|
||||
|
||||
- Likely storage split:
|
||||
- PRD/board/manifest in git-backed docs
|
||||
- mission/session state in the Mosaic data layer
|
||||
- runtime health in queue/session state
|
||||
- Worktrees and long-lived agent working directories should live under `/src/<repo>-worktrees` rather than `/tmp` so they sit on the larger persistent drive and survive longer-running missions.
|
||||
- The coordinator needs a stable session identity, even if the active session changes.
|
||||
- Task dependencies must be enforced so workers do not start early.
|
||||
- The handoff packet should include the top 3 immediate actions and the strongest constraints.
|
||||
- Rotation triggers should be configurable per profile or per mission.
|
||||
- The initial version can be file-first, with dashboard sync added later.
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
- A mission can rotate sessions without losing the active goal.
|
||||
- A new session can resume from the latest handoff in under one turn.
|
||||
- Board tasks remain aligned to PRD user stories.
|
||||
- Short-cycling sessions are replaced before repeated compaction harms quality.
|
||||
- Operators can find mission state without spelunking across multiple chat logs.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. What should the canonical mission ID format be?
|
||||
2. Should the board live only in git, or also in the database?
|
||||
3. Should rotation be automatic by default, or opt-in per mission?
|
||||
4. What should the short-cycle threshold be initially?
|
||||
5. Should handoffs be pure text, structured JSON, or both?
|
||||
6. Which CLI command should be the primary mission entrypoint: `mosaic mission`, `mosaic coord`, or `mosaic prdy`?
|
||||
@@ -0,0 +1,113 @@
|
||||
# Tasks — Mosaic Mission Control Plane
|
||||
|
||||
> Single-writer: orchestrator only. Workers read but never modify.
|
||||
>
|
||||
> **Mission:** mission-control-plane-20260506
|
||||
> **Schema:** `| id | status | description | issue | agent | branch | depends_on | estimate | notes |`
|
||||
> **Status values:** `not-started` | `in-progress` | `done` | `blocked` | `failed` | `needs-qa`
|
||||
> **Agent values:** `codex` | `glm-5.1` | `haiku` | `sonnet` | `opus` | `—` (auto)
|
||||
>
|
||||
> Scope: this file decomposes the combined PRD / mission / board workflow into atomized tasks.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 1 — PRD + mission schema foundation
|
||||
|
||||
Goal: create the durable doc structure and the minimal mission metadata needed to keep PRD, board, and mission aligned.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ----------- | -------------------------------------------------------------------------------------------------------- | ----- | ------ | ----------------------------- | ------------------ | -------- | ------------------------------------------- |
|
||||
| MC-01-01 | not-started | Write `docs/mission-control/PRD.md` with goals, non-goals, functional requirements, and success metrics. | — | sonnet | docs/mission-control-prd | — | 5K | Human-readable PRD becomes the spec anchor. |
|
||||
| MC-01-02 | not-started | Write `docs/mission-control/BOARD.md` as a decision board for scope, priority, and open questions. | — | haiku | docs/mission-control-board | MC-01-01 | 3K | Keeps discussion separate from the spec. |
|
||||
| MC-01-03 | not-started | Write `docs/mission-control/MISSION-MANIFEST.md` linking PRD, board, tasks, and mission identity. | — | sonnet | docs/mission-control-manifest | MC-01-01, MC-01-02 | 4K | Durable mission root object. |
|
||||
| MC-01-04 | not-started | Write `docs/mission-control/TASKS.md` with the atomized execution plan and dependency graph. | — | sonnet | docs/mission-control-tasks | MC-01-03 | 4K | Board-backed execution plan. |
|
||||
|
||||
**Milestone 1 estimate:** ~16K tokens
|
||||
|
||||
---
|
||||
|
||||
## Milestone 2 — Mission runtime model
|
||||
|
||||
Goal: make missions first-class runtime objects that can survive session restarts and compaction.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ----- | -------------------------------------- | ---------------------------------- | -------- | ------------------------------------------ | ---------------------------------------------------- |
|
||||
| MC-02-01 | not-started | Define mission schema in the data layer: mission ID, goal, phase, PRD path, board path, active session ID, last handoff, and churn score. | — | codex | feat/mission-control-schema | MC-01-03 | 6K | This is the durable root state. |
|
||||
| MC-02-02 | not-started | Add mission read/write services to `packages/coord` so the coordinator can load and persist mission state. | — | codex | feat/mission-control-coord-store | MC-02-01 | 6K | Keep storage simple and explicit. |
|
||||
| MC-02-03 | not-started | Add mission status reporting to `mosaic mission` and `mosaic coord status`. | — | codex | feat/mission-control-status-cli | MC-02-02 | 4K | Operators need one obvious status command. |
|
||||
| MC-02-04 | not-started | Add tests for mission persistence and recovery after restart. | — | haiku | feat/mission-control-persistence-tests | MC-02-02 | 4K | Verify mission survives process churn. |
|
||||
| | MC-02-05 | done | Add a worktree-root convention to the mission runtime notes and startup guidance so agents prefer `/src/<repo>-worktrees` over `/tmp`. | — | haiku | docs/mission-control-worktree-root | MC-01-03 | 3K | Keep long-lived work on the larger persistent drive. |
|
||||
|
||||
**Milestone 2 estimate:** ~20K tokens
|
||||
|
||||
---
|
||||
|
||||
## Milestone 3 — Board atomization and task linkage
|
||||
|
||||
Goal: derive assignable tasks from the PRD and keep them linked to mission state.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ----------- | ------------------------------------------------------------------------------------------- | ----- | ------ | -------------------------------- | ------------------ | -------- | ------------------------------------------- |
|
||||
| MC-03-01 | not-started | Add a PRD-to-task decomposition rule set: every user story maps to one or more board tasks. | — | sonnet | feat/mission-control-decompose | MC-01-01 | 5K | Start simple and deterministic. |
|
||||
| MC-03-02 | not-started | Implement board generation from the PRD in a machine-readable format. | — | codex | feat/mission-control-board-gen | MC-03-01 | 6K | Output should be usable by the coordinator. |
|
||||
| MC-03-03 | not-started | Add dependency validation so tasks cannot start before parent tasks complete. | — | codex | feat/mission-control-deps | MC-03-02 | 5K | Enforces ordering. |
|
||||
| MC-03-04 | not-started | Add review-task support so a mission cannot close without a reviewer step. | — | sonnet | feat/mission-control-review-gate | MC-03-03 | 4K | Preserves quality. |
|
||||
| MC-03-05 | not-started | Add tests proving the board stays traceable back to the PRD user stories. | — | haiku | feat/mission-control-trace-tests | MC-03-02, MC-03-03 | 4K | Traceability is the point. |
|
||||
|
||||
**Milestone 3 estimate:** ~24K tokens
|
||||
|
||||
---
|
||||
|
||||
## Milestone 4 — Short-cycle detector and rotation engine
|
||||
|
||||
Goal: detect when a session is stuck and rotate to a fresh session before quality falls off.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------- | ----- | ------ | ----------------------------------- | ---------- | -------- | ---------------------------------------------- |
|
||||
| MC-04-01 | not-started | Define churn signals: repeated compaction, identical tool loops, repeated permission prompts, and no progress across several turns. | — | sonnet | feat/mission-control-churn-signals | MC-02-01 | 4K | Keep the rules explicit. |
|
||||
| MC-04-02 | not-started | Implement churn scoring in the coordinator with configurable thresholds. | — | codex | feat/mission-control-churn-score | MC-04-01 | 6K | Weighted score makes tuning easier. |
|
||||
| MC-04-03 | not-started | Implement automatic session rotation when churn crosses the threshold. | — | codex | feat/mission-control-rotate-session | MC-04-02 | 6K | The session is disposable; the mission is not. |
|
||||
| MC-04-04 | not-started | Add tests for rotation triggers and for avoiding premature rotation. | — | haiku | feat/mission-control-rotation-tests | MC-04-03 | 4K | Prevent flapping. |
|
||||
|
||||
**Milestone 4 estimate:** ~20K tokens
|
||||
|
||||
---
|
||||
|
||||
## Milestone 5 — Handoff generation and re-entry
|
||||
|
||||
Goal: preserve the best context from the old session and inject it into the new session cleanly.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ----------- | -------------------------------------------------------------------------------------------------------------------- | ----- | ------ | ----------------------------------- | ------------------ | -------- | ---------------------------------------- |
|
||||
| MC-05-01 | not-started | Define the handoff packet schema: mission ID, session ID, completed work, blockers, next 3 actions, and constraints. | — | sonnet | feat/mission-control-handoff-schema | MC-02-01 | 4K | Keep it compact and structured. |
|
||||
| MC-05-02 | not-started | Implement handoff packet writing during rotation. | — | codex | feat/mission-control-handoff-write | MC-05-01, MC-04-03 | 5K | Persist before the old session exits. |
|
||||
| MC-05-03 | not-started | Implement handoff packet loading at session startup. | — | codex | feat/mission-control-handoff-load | MC-05-01, MC-04-03 | 5K | New session should know the next action. |
|
||||
| MC-05-04 | not-started | Add tests proving a rotated session can continue the mission without manual re-prompting. | — | haiku | feat/mission-control-handoff-tests | MC-05-02, MC-05-03 | 4K | Resume quality is the key metric. |
|
||||
|
||||
**Milestone 5 estimate:** ~18K tokens
|
||||
|
||||
---
|
||||
|
||||
## Milestone 6 — Operator surface and E2E validation
|
||||
|
||||
Goal: expose the whole workflow through commands and verify it end-to-end.
|
||||
|
||||
| id | status | description | issue | agent | branch | depends_on | estimate | notes |
|
||||
| -------- | ----------- | --------------------------------------------------------------------------------------------------------- | ----- | ------ | -------------------------------- | ------------------ | -------- | -------------------------------------------- |
|
||||
| MC-06-01 | not-started | Add a CLI command to inspect the active mission, PRD path, board path, task statuses, and latest handoff. | — | codex | feat/mission-control-inspect-cli | MC-02-03, MC-05-03 | 5K | One place to inspect the whole stack. |
|
||||
| MC-06-02 | not-started | Add a compact dashboard or TUI summary view for mission health. | — | codex | feat/mission-control-summary-ui | MC-06-01 | 6K | Nice to have, but not before the core works. |
|
||||
| MC-06-03 | not-started | Build an E2E harness that simulates compaction / rotation and verifies the mission can continue. | — | sonnet | feat/mission-control-e2e-harness | MC-04-03, MC-05-03 | 8K | This is the proof that the design works. |
|
||||
| MC-06-04 | not-started | Add final docs for operators explaining how PRD, mission, and board fit together. | — | haiku | feat/mission-control-ops-docs | MC-06-03 | 4K | Make it usable by humans. |
|
||||
| MC-06-05 | not-started | Consolidate review findings and close the mission with a release note. | — | sonnet | chore/mission-control-close | MC-06-04 | 3K | Only after the E2E passes. |
|
||||
|
||||
**Milestone 6 estimate:** ~26K tokens
|
||||
|
||||
---
|
||||
|
||||
## Execution Notes
|
||||
|
||||
- `sonnet` is best for planning, decomposition, and the review-gate tasks.
|
||||
- `codex` is best for schema, coordinator, and CLI implementation.
|
||||
- `haiku` is best for validation, traceability checks, and docs.
|
||||
- The first implementation pass should stay file-first and keep the runtime state thin.
|
||||
- The mission should not close until the PRD, board, mission manifest, and E2E harness all agree.
|
||||
@@ -0,0 +1,36 @@
|
||||
# Documentation Completion Checklist — Native Kanban/SOT Canon
|
||||
|
||||
**Tracking:** Mosaic Stack issue #751
|
||||
**Scope:** Requirements and contract publication only; runtime implementation follows in separate slices.
|
||||
|
||||
## Required artifacts
|
||||
|
||||
- [x] Project `docs/PRD.md` exists; the workstream requirements refine its task/project-management scope.
|
||||
- [x] Canonical workstream requirements published at `docs/requirements/native-kanban-sot.md`.
|
||||
- [x] Mission manifest, task decomposition, frozen shared contract, and typed contract declarations included.
|
||||
- [x] `docs/SITEMAP.md` updated.
|
||||
- [x] Independent initial review and final GO report stored under `docs/reports/native-kanban-sot/`.
|
||||
- [x] Task scratchpad stored under `docs/scratchpads/`.
|
||||
- [ ] User/Admin/Developer guides — N/A for canon-only publication; required in implementation slices that change behavior or operations.
|
||||
- [ ] OpenAPI and endpoint index — N/A until KBN-105 freezes implementation-ready endpoint contracts.
|
||||
|
||||
## Structural and root hygiene
|
||||
|
||||
- [x] Canonical requirements are under `docs/requirements/`.
|
||||
- [x] Workstream artifacts are under `docs/native-kanban-sot/`.
|
||||
- [x] Review reports are under `docs/reports/native-kanban-sot/`.
|
||||
- [x] No new unscoped document was added to the `docs/` root.
|
||||
- [x] Root mission/task rollups link to the workstream.
|
||||
|
||||
## Review gate
|
||||
|
||||
- [x] Author and independent reviewer are different agents.
|
||||
- [x] KCR-001–016 closure was independently verified.
|
||||
- [x] Ultron final gate returned GO with zero BLOCKER/HIGH findings.
|
||||
- [x] Formatter, lint, typecheck, strict contract TypeScript, link, scope, and invariant publication validation passed in the current Stack toolchain.
|
||||
- [ ] PR review, CI, squash merge, and issue closure remain required before publication completion.
|
||||
|
||||
## Publishing
|
||||
|
||||
- [x] Canonical source remains in-repository.
|
||||
- [x] No external publishing platform is required for this internal architecture contract.
|
||||
@@ -0,0 +1,67 @@
|
||||
# Native Kanban/SOT Canon
|
||||
|
||||
**Status:** KCR-001–016 independently cleared; KBN-101 rc.16 current generic storage-wrapper authority remediation awaits independent exact-head re-review under issue [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771)
|
||||
**Date:** 2026-07-14
|
||||
**Implementation hold:** no feature implementation starts until this canon is squash-merged to `main` with terminal-green CI; after merge, every slice remains held until its KBN prerequisite graph is satisfied.
|
||||
|
||||
## Artifacts
|
||||
|
||||
| Artifact | Purpose |
|
||||
| -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| [Canonical requirements](../requirements/native-kanban-sot.md) | Canonical P0–P3 requirements, all seven ratified decisions, fixed invariants, thin MVP, recovery tiers, non-goals, and per-requirement acceptance criteria |
|
||||
| [`MISSION-MANIFEST.md`](./MISSION-MANIFEST.md) | Mission/authority boundaries, exact role chain, gate model, mandatory SecReview triggers, Certifier final/no-merge rule, and collision-free slice ownership |
|
||||
| [`TASKS.md`](./TASKS.md) | Dependency-ordered, bounded P0–P3 slices with IN/OUT scope, dependencies, shared contracts, file ownership, evidence, and USC coder2/3/4/5 parallelization |
|
||||
| [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md) | rc.16 direct-Drizzle current storage-wrapper hold: legacy N-1/uncertified/non-operative pending -02/-03/-06/-08; exact README commented/user-guide executable forms fail before masking and source-consistency rejects runner-delegation copy; held future bootstrap → TLS/roles → run → verify → readiness; plus prior production boundary, pgvector owner, attestation, inventory, manifests, DDL classifier, TLS/bootstrap, activation, and certification contract; foundation prerequisite of KBN-100 and real-role gate before KBN-105 |
|
||||
| [`KBN-101-ENVELOPE-A.md`](./KBN-101-ENVELOPE-A.md) | KBN-101 Envelope A (v6) — RATIFIED, part of the frozen SSOT: rc.20 declarative sink-RBAC + per-role connection-selection + RLS `WITH CHECK`/`USING` write-source + `FORCE ROW LEVEL SECURITY` + sink-resident `task_status_write_override`; adds owner card KBN-101-10 + responsibility-widenings; authority Jason B1 + Mos OPTION A/Q1/Q2 |
|
||||
| [`SHARED-CONTRACT.md`](./SHARED-CONTRACT.md) | Remediated v1 integration contract: proof authority, exact failures/routes/DTOs/MCP ownership, concrete current-main field migration map, relational invariants, Coordinator split, recovery delivery |
|
||||
| [`contracts/kanban-schema.v1.ts`](./contracts/kanban-schema.v1.ts) | Drizzle target declarations including exact owner/principal membership, project congruence, tags/archive, proposals, persisted assignments, monotonic fences, durable retry, immutable evidence/audit |
|
||||
| [`contracts/mechanical-coordinator.v1.ts`](./contracts/mechanical-coordinator.v1.ts) | Pure snapshot decision engine separated from persistence/service adapter; ID-bound approvals, bigint-safe fences, durable retry/quarantine, artifact-backed checkpoints, exact failures |
|
||||
| [`contracts/health-state.v1.ts`](./contracts/health-state.v1.ts) | Discriminated public health, separate branded transaction-local write proof, and non-overlapping denial/transport/version-conflict mappings |
|
||||
| [`contracts/recovery-posture.v1.ts`](./contracts/recovery-posture.v1.ts) | Provider-neutral shape schema plus normative runtime refinement, cross-field constraints, and Lite/Standard/High-assurance defaults |
|
||||
| [`tsconfig.json`](./tsconfig.json) | Strict no-emit project scope for linting and compiling the four frozen TypeScript contracts against the current Stack Drizzle declarations |
|
||||
| [`DOCUMENTATION-CHECKLIST.md`](./DOCUMENTATION-CHECKLIST.md) | Publication documentation gate and implementation-slice deferrals |
|
||||
| [KBN-101 exact-head security review](../reports/native-kanban-sot/kbn-101-contract-security-review-82ce325.md) | Historical `da742ca` REQUEST CHANGES report retained as prior closure evidence; rc.16 awaits independent exact-head re-review after closing the current generic storage-wrapper authority HIGH finding |
|
||||
| [Initial independent review](../reports/native-kanban-sot/canon-initial-review-no-go.md) | KCR-001–016 findings that blocked the first draft |
|
||||
| [Final independent re-review](../reports/native-kanban-sot/canon-final-rereview-go.md) | Closure matrix, reproducible validation evidence, and GO verdict |
|
||||
| [Ultron final gate](../reports/native-kanban-sot/ultron-final-go.md) | Final requirements, authority, schema, migration, recovery, decomposition, and evidence review GO |
|
||||
|
||||
## Recommended USC lane partition
|
||||
|
||||
| Lane | Natural seam | Exclusive ownership |
|
||||
| ---------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **coder2** | Schema + migrations + recovery slice | Unified Drizzle schema, migration SQL/meta/journal/tests, then recovery parser/mechanism/runbook files |
|
||||
| **coder3** | Domain + Gateway + MCP server | Workspace-safe repositories, DTOs/controllers/services, exact `apps/gateway/src/mcp/**` files, health proof, proposals, Coordinator persistence adapter |
|
||||
| **coder4** | Pure Coordinator + tooling | `packages/coord` mechanical engine, CLI/MCP consumers, generated projection, one-way importer and cutover tooling; lane-serialized internally |
|
||||
| **coder5** | Web | Tasks/Projects Kanban/List/detail and later Coordinator/migration-review UI |
|
||||
| **Mos** | Serialized integration | Canon publication, frozen-contract changes, shared-root/exports, integration gates, merge authority |
|
||||
|
||||
The safe order is KBN-010 → KBN-101 foundation → KBN-100 → KBN-101 deployed-role immutable-operation certificate → KBN-105, then coder3 Gateway/MCP server, coder4 CLI/projection, coder5 web, and coder2 recovery can proceed on disjoint files. KBN-100 is blocked on the KBN-101 foundation; real deployed-role certification—not synthetic test roles—is required before KBN-105. coder4 then runs pure Coordinator → importer → cutover tooling serially. No two active slices edit the same files.
|
||||
|
||||
## Recovery defaults
|
||||
|
||||
| Tier | RPO / RTO | WAL / PITR | Base backup | Restore / break-glass | Off-cluster |
|
||||
| -------------- | ------------ | ------------------- | ----------- | ----------------------- | ----------------------------------------------- |
|
||||
| Lite | 24h / 24h | disabled / disabled | daily | quarterly / annual | encrypted separate target |
|
||||
| Standard | 1h / 8h | q15m / 14d | daily | quarterly / semiannual | encrypted separate object storage |
|
||||
| High-assurance | **15m / 4h** | **q5m / 35d** | **daily** | **monthly / quarterly** | **encrypted base+WAL, separate failure domain** |
|
||||
|
||||
These knobs affect recovery posture only. PostgreSQL remains the sole writable SOT in every tier. Fail-closed writes, generated-file non-authority, attributable post-recovery proposals, non-LLM Coordinator limits, and Certifier final-gate/no-merge authority are fixed for every tier.
|
||||
|
||||
## Non-blocking implementation sub-decisions for Mos
|
||||
|
||||
The source plan and ratified seven decisions resolve all build-blocking product choices. The following implementation-local selections remain for the owning slices/Mos and must not weaken v1:
|
||||
|
||||
1. Exact PostgreSQL write-health probe SQL and bounded proof lifetime; authority and failures are frozen.
|
||||
2. Dependency-cycle serialization mechanism (recursive CTE plus transaction/advisory lock or equivalent); required behavior is frozen.
|
||||
3. Whether RLS lands in the first migration or immediately after the tested session-context pattern; workspace constraints/repository authorization are required from migration one.
|
||||
4. Concrete off-cluster backup provider/bucket and selected production recovery tier; High-assurance minima are frozen if selected.
|
||||
5. Cutover reconciliation thresholds and stabilization duration, to be owner-approved before P3 execution.
|
||||
|
||||
None authorizes a second writer, dual sync, LLM scheduling, Coordinator gate waiver/merge, or Certifier merge authority.
|
||||
|
||||
## Publication validation evidence
|
||||
|
||||
- Concrete TypeScript contracts are formatted with repository Prettier.
|
||||
- All four contracts pass strict TypeScript no-emit checking against the current Stack Drizzle toolchain.
|
||||
- Contract remediation and KCR-001–016 traceability are recorded in the issue scratchpad and linked review reports.
|
||||
- Independent re-review returned GO with KCR-001–016 closed; implementation remains held until canon merge and the dependency-ordered KBN prerequisites complete.
|
||||
@@ -0,0 +1,415 @@
|
||||
# KBN-010 — Threat, Authorization, and Constraint-Impact Gate
|
||||
|
||||
- **Issue:** [#753](https://git.mosaicstack.dev/mosaicstack/stack/issues/753)
|
||||
- **Gate status:** **PASS / GO**
|
||||
- **Reviewed baseline:** `origin/main` at `49e8a54` (2026-07-14)
|
||||
- **Frozen target:** `SHARED-CONTRACT.md` v1.0.0-rc.4 and `contracts/*.v1.ts`
|
||||
- **Disposition input:** contract commit `3f6a3387b419eb99453ee10dd25ba888faaab0b5`, tree `7ebab8fa530a7180036928cea9527f808548aa14`
|
||||
- **Scope:** documentation and future-test planning only; no runtime, schema, migration, API, configuration, dependency, CI, or deployment change
|
||||
|
||||
## 1. Decision
|
||||
|
||||
KBN-010 is **PASS / GO** against frozen contract rc.4. The original rc.3 finding remains historical detection evidence:
|
||||
|
||||
- **KBN010-SI-001 — rc.3 invalid mission composite-FK candidate key.** At rc.3, `missionsV1` declared a primary key on `id` and a unique key on `(workspace_id, project_id, id)`, but not a candidate key on `(workspace_id, id)`. Both `artifacts_workspace_mission_fk` and `approval_decisions_workspace_mission_fk` referenced exactly `(missions.workspace_id, missions.id)`. PostgreSQL requires the referenced column list of a foreign key to match a non-partial unique/primary candidate key; uniqueness of `id` alone did not satisfy that two-column reference. The rc.3 DDL was therefore invalid, and KBN-010 correctly blocked it.
|
||||
|
||||
Contract rc.4 resolves SI-001 by adding the non-partial `missions_workspace_id_uidx` candidate key on `(workspace_id, id)` while retaining the global `id` primary key and the project-congruent `(workspace_id, project_id, id)` key. Both polymorphic child FKs retain their exact workspace-safe ordered columns and `ON DELETE RESTRICT`; no target, tenancy, project-congruence, exactly-one-target, N-1, rollback, no-cascade, identity, approval, or fencing authority is weakened.
|
||||
|
||||
Independent Homelab non-author schema/security review returned **APPROVE** for the exact rc.4 commit/tree/content and found no collision with #757 connector fencing. SI-001 has no unresolved contract/schema-design impact.
|
||||
|
||||
This GO completes the KBN-010 analysis/review prerequisite only. It does **not** claim that runtime schema or migration DDL exists. KBN-100 remains held and may be released only after this PR squash-merges, the merged change reaches terminal-green CI on `main`, and issue #753 closes.
|
||||
|
||||
### 1.1 Independent rc.4 evidence identity
|
||||
|
||||
- **Commit:** `3f6a3387b419eb99453ee10dd25ba888faaab0b5`
|
||||
- **Tree:** `7ebab8fa530a7180036928cea9527f808548aa14`
|
||||
- **Stable full-index SHA-256:** `6b40a76265c4f3e6d1d30a7f262a2dd16e0d51997e99c146b59f527e6524cd42`
|
||||
- **Stable patch-id:** `058cf98026fcd1043703c866aee047c8bb144740`
|
||||
- **Verdict:** Homelab independent non-author schema/security review **APPROVE**.
|
||||
- **Reviewed conclusions:** the candidate key repairs both dependent FKs; tenant safety, polymorphic exactly-one-target semantics, RESTRICT/no-cascade behavior, and N-1/rollback semantics remain valid; #757 uses separate tables/indexes/FKs/identity/fence authority and has no collision.
|
||||
|
||||
A command-rendered patch SHA may differ when Git rendering options, headers, or command form differ. That rendering digest is non-authoritative. Canonical review identity is the Git commit object plus tree and exact file content; the stable full-index digest and stable patch-id above are corroborating identities.
|
||||
|
||||
## 2. Method and trust boundaries
|
||||
|
||||
### 2.1 Inputs inspected
|
||||
|
||||
- Canonical requirements: `docs/requirements/native-kanban-sot.md`.
|
||||
- Workstream manifest and read-only task plan.
|
||||
- Frozen health, schema, Mechanical Coordinator, and recovery contracts in full.
|
||||
- Actual current-main schema, Better Auth guard/scope helpers, project/task/mission/team controllers and repositories, fleet backlog, and `TASKS.md` parser/writer.
|
||||
- Issue #753 through the Mosaic provider wrapper.
|
||||
|
||||
### 2.2 Current-main exposure that the target must replace, not inherit
|
||||
|
||||
| Current-main fact | Constraint on future implementation |
|
||||
| --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Teams are global; projects, missions, tasks, agents, and fleet backlog have no `workspace_id`. | KBN-100 must add the workspace boundary and KBN-110 must query by server-derived workspace in every repository operation. |
|
||||
| `AuthGuard` authenticates a Better Auth user, while `scopeFromUser` falls back through optional tenant/team/org claims and finally user ID. | Kanban tenancy must derive from an authenticated **active workspace membership**, not this compatibility fallback or caller data. |
|
||||
| Team list/get/member endpoints return global team data to any authenticated user. | New Kanban endpoints must use a uniform no-oracle denial and must not reuse global team lookup as authorization. |
|
||||
| Project/task repositories load and mutate by bare IDs; controller checks are separate and sometimes distinguish not-found from forbidden. | Workspace predicates and authorization must be inside the authoritative transaction/repository command path. |
|
||||
| Tasks can have nullable project/mission links, free-text assignee, JSON tags, no aggregate version, and no fence. | Expand/backfill/quarantine must precede NOT NULL/composite constraints; new commands cannot trust legacy fields. |
|
||||
| `mission_tasks.status` is a second status writer. | Pre-expand must prohibit it as a write source and later retire it only after N-1 evidence. |
|
||||
| Fleet `backlog` has global JSON dependencies and TTL claims without workspace, assignment, approval, session, or fencing. | It must be frozen and imported as non-dispatching shadow data; it cannot be adapted into the canonical lease path. |
|
||||
| `packages/coord/src/tasks-file.ts` parses and mutates `TASKS.md`. | KBN-120 must replace production use with generated, read-only projection code and prove there is no import/mutation path. |
|
||||
| No Kanban transaction-local health proof, semantic audit/event chain, change proposals, canonical outbox, approval binding, or fenced lease model exists. | These are new frozen invariants, not behaviors that may be inferred from current endpoints. |
|
||||
|
||||
## 3. Authorization matrix
|
||||
|
||||
The exact route/DTO freeze belongs to KBN-105. This matrix fixes the minimum authorization behavior that freeze and later implementation must preserve.
|
||||
|
||||
| Principal/state | Permitted authority | Required authoritative checks | Explicit denials |
|
||||
| -------------------------------------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
|
||||
| Unauthenticated caller | Public health observation only, if deployment exposes it | Health DTO validation; no proof field accepted | All canonical reads/mutations; health observation never authorizes a write |
|
||||
| Active workspace `owner`/`admin` user | Policy-allowed workspace administration and domain commands | Better Auth session; active membership; server-derived workspace; command-family role; expected version/idempotency | Foreign workspace, suspended workspace, revoked membership, caller workspace override |
|
||||
| Active workspace `member` user | Policy-allowed project/task/proposal commands | Active membership plus project/team capability and target checks in the same transaction | Admin, approval, purge, service-only Coordinator, and unrelated project commands |
|
||||
| Active workspace `auditor` user | Workspace-scoped reads and audit/evidence inspection | Active membership and read capability | Every mutation, approval, lease, token issuance, purge |
|
||||
| Active workspace `service` identity | Only explicitly issued command families | Credential maps to workspace+agent+session; agent enabled; session live; role/capability allowlist; token expiry/audience; DB recheck per command | Raw DB credentials, user/admin fallback, cross-workspace scope, command families absent from token and registry |
|
||||
| Enabled agent with live session | Agent commands matching its declared and policy-approved specialist role/capabilities | Exact workspace+agent+session binding, heartbeat/state, assignment target, lease, current decimal-string fence | Ended/offline/degraded session where policy disallows; disabled agent; another assignment/session/fence |
|
||||
| Mechanical Coordinator engine | Pure eligibility/order/expiry decisions from immutable snapshots | Complete workspace-local snapshot and policy revision | Authentication, ID loading, SQL, proof minting, scope invention, approval, certification, merge |
|
||||
| Coordinator persistence service | Service-only assignment/lease/checkpoint/recovery commands | Fresh transaction-local proof; locks; current assignment/approval/task/session/policy/fence | Public/user proof-by-value, stale approval/policy, direct completion/certification/merge |
|
||||
| Reviewer/SecReview/Certifier | Attributable evidence decisions allowed by gate policy | Active authority, author differs from reviewer, mandatory SecReview classification, immutable artifacts | Self-review; missing evidence; Certifier merge/issue-close/release |
|
||||
| Break-glass retention operator | Narrow, time-bounded purge procedure only | Separate break-glass authority, reason, scope, approvals, immutable pre-purge evidence, semantic audit, post-action reconciliation | Normal application role DELETE/UPDATE, bulk unscoped purge, unaudited hard delete |
|
||||
| Revoked/expired/disabled identity or ended session | None beyond policy-permitted public observation | Revocation/lifecycle checked from PostgreSQL on every command | Cached token/Valkey state cannot preserve authority |
|
||||
|
||||
**No-oracle rule:** authentication may return 401, but once authenticated, a foreign-workspace, nonexistent, inaccessible, or wrong-project identifier must follow the one KBN-105-frozen 404/403 policy with the same response shape and no foreign metadata, timing-derived detail, or WebSocket/MCP discrepancy.
|
||||
|
||||
## 4. Threat matrix
|
||||
|
||||
Every disposition is against the frozen target, not a claim about current-main behavior.
|
||||
|
||||
| ID | Attacker or failure | Asset | Precondition and abuse path | Frozen preventive/detective control | Required schema/API/negative-test evidence | Future owner | Residual risk | Disposition |
|
||||
| --- | --------------------------------------------------------------------------------------- | --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
|
||||
| T01 | Authenticated user supplies a foreign workspace/resource ID | Tenant confidentiality and integrity | Caller knows or guesses project/task/mission/team IDs and probes REST, MCP, WebSocket, repository, or Coordinator paths | `workspace_id` on every canonical row; composite relations; server-derived tenant; uniform no-oracle denial | Composite FK/unique DDL; every repository predicate includes workspace; N100-01/02, N110-01..05, N130-01 | KBN-100, 105, 110, 130 | Timing/volume side channels require operational review | Controlled after evidence |
|
||||
| T02 | Revoked or inactive member retains an old session | Ownership and mutation authority | Authentication remains valid after workspace membership revocation | Active membership rechecked in the authoritative transaction for owners, principals, proposers, and decision actors | Active/inactive membership fixtures; N100-03, N110-06/07; no cached membership authority | KBN-100, 110 | Better Auth session may remain valid for unrelated features | Controlled after evidence |
|
||||
| T03 | User joins/forges a team relation outside its workspace | Team-owned projects and tasks | Global-current-main team behavior or a stale membership is reused | Team is intra-workspace only; workspace/team composites; active workspace membership precedes team authorization | Cross-workspace team/member/owner insert and command denials; N100-04/05, N110-08 | KBN-100, 110 | Team-role policy mistakes remain possible | Controlled after evidence |
|
||||
| T04 | Same-workspace IDs from a different project are combined | Planning hierarchy integrity | Valid mission/milestone/parent/current-milestone UUIDs are substituted | Project-congruent composite relations and serialized hierarchy validation | Mission/milestone/parent/current milestone mismatch and parent-cycle tests; N100-06..10, N110-09 | KBN-100, 110 | Deep hierarchy checks can be expensive | Controlled after evidence |
|
||||
| T05 | Foreign or unrelated evidence/link/artifact IDs are attached | Review and audit truth | Caller has a valid same-workspace or foreign artifact UUID | Workspace-aware joins; immutable artifact digest/revision; semantic same-target validation in authoritative transaction | Mixed-workspace and same-workspace wrong-task/mission checkpoint/approval evidence tests; N100-11..14, N210-15/16 | KBN-100, 110, 210 | Same-workspace semantic validation is application-enforced | Controlled after evidence |
|
||||
| T06 | Stolen, over-scoped, or replayed service token | Coordinator and task mutation authority | Service credential is accepted as admin/user or claims are trusted without DB state | Command-family least privilege; agent/session workspace binding; no raw DB credentials; enabled/live state checked per command | Auth registry fixtures prove audience/expiry/role/capability; revoked agent and ended session denials; N105-01, N110-10..13, N210-01/02 | KBN-105, 110, 210 | Credential theft until expiry/revocation check | Controlled after evidence |
|
||||
| T07 | Caller forges public `healthy` or replays a stale health response | Sole-writer/fail-closed invariant | Public health body or caller field reaches mutation context | Public DTO is observation only; public DTOs reject proof/health fields; Gateway mints internal proof after live PG transaction probe | Contradictory union and forbidden-field tests; N105-02, N110-14..17 | KBN-105, 110, 140 | Health endpoint can still be used for reconnaissance | Controlled after evidence |
|
||||
| T08 | Internal stale, wrong-policy, or wrong-transaction proof is reused | Transaction integrity | A branded value leaks or an adapter fails to revalidate it | Non-exported brand; transaction identity, `checkedAt <= now < validUntil`, and policy revision revalidated immediately before mutation | Wrong transaction, expiry boundary, future timestamp, policy mismatch, commit-after-expiry tests; N110-18..22 | KBN-110, 140 | In-process code can bypass TypeScript; runtime checks are mandatory | Controlled after evidence |
|
||||
| T09 | DB/transport uncertainty is mislabeled as deliberate denial or conflict | Safe retry and exactly-once result | Timeout occurs before/after commit and client changes key or retries 503 | Exact 503/502/504/timeout/409 union; unknown outcome retries only with same idempotency key | Exhaustive fixture mapping and commit-before-timeout replay; N105-03, N110-23..27, N120-01/02 | KBN-105, 110, 120, 140 | External client may ignore retry rules | Controlled after evidence |
|
||||
| T10 | Assignment payload forges task version, target agent/session, role, expiry, or proposer | Work routing authority | Lease service trusts command DTO rather than persisted assignment | Persisted assignment identity; exactly-one principal/proposer; exact agent/session composite; acquire accepts IDs then reloads+locks | Cross-workspace and same-workspace target substitutions, stale task version, invalid role, expired assignment; N100-15..18, N210-03..08 | KBN-100, 200, 210 | Compromised authorized proposer can make harmful proposals | Controlled by approval/audit |
|
||||
| T11 | Approval proof is forged by value or borrowed from another assignment | Gate integrity | Caller submits `approved=true`, unrelated decision ID, stale policy, or self-approval | Relational approval bound to assignment; lock/reload; policy revision; author≠reviewer and mandatory SecReview | No proof-by-value DTO; wrong assignment/task/workspace/policy/actor/decision tests; N105-04, N210-09..14, N230-01 | KBN-105, 210, 230 | Colluding principals remain an organizational risk | Controlled after evidence |
|
||||
| T12 | Revoked policy or expired proposal/assignment is raced against lease acquisition | Routing policy | Approval and lease transactions do not lock/revalidate current rows | Lock assignment, approval, task, target session; compare current policy and expiry inside fresh-proof transaction | Concurrent revoke/expire/acquire tests with one valid terminal result; N210-17..19 | KBN-210, 230 | Clock skew if DB time is not canonical | Controlled after evidence |
|
||||
| T13 | Stale worker sends ack/heartbeat/checkpoint/review after reassignment | Canonical task and evidence state | Old process retains task/session IDs | Task-row-locked atomic monotonic bigint fence; every worker command carries exact lease/session/fence | Lower, expired, future, and other-task fences denied; old worker loses after new lease; N100-19/20, N210-20..24 | KBN-100, 210, 230 | Signed bigint exhaustion is theoretical | Controlled after evidence |
|
||||
| T14 | JavaScript precision truncates a fence | Stale-worker exclusion | bigint token is serialized as number above `2^53-1` | Drizzle bigint and decimal-string wire type only | `9007199254740993` and near-`int8` boundary round trips; numeric JSON rejected; N105-05, N210-25 | KBN-105, 210 | Nonconforming external clients | Controlled after evidence |
|
||||
| T15 | Checkpoint/evidence from another lease/task/session is submitted | Recovery and certification evidence | Same-workspace valid IDs are mixed | Exact lease composite binds workspace+task+assignment/session+fence; checkpoint composite binds lease+fence; evidence join plus semantic artifact-owner check | Same-workspace mismatched task/assignment/lease/session/checkpoint/artifact tests; N100-21..23, N210-26..31 | KBN-100, 210 | Artifact URI target may disappear outside DB | Controlled with digest/retention |
|
||||
| T16 | Outage note or pending/rejected proposal mutates/orders work | Sole SOT and gate integrity | Importer/UI treats note/proposal as task state | Proposals are inert; only explicit accept invokes normal typed command after recovery | Row/outbox/task counts unchanged for pending/rejected; no readiness/dependency/lease effect; N110-28..31 | KBN-110, 140 | Humans may act outside Mosaic operationally | Accepted as attributable residual |
|
||||
| T17 | Submission event is missing, foreign, or for another proposal | Proposal audit chain | Caller supplies an existing event UUID | Preallocated proposal ID; event-first same transaction; workspace composite FK; exact event type/aggregate/version semantic check | Missing/foreign/wrong-type/wrong-proposal event rolls back event+proposal; N100-24/25, N110-32..36 | KBN-100, 110 | Semantic checks are transaction code, not only FK | Controlled after evidence |
|
||||
| T18 | Acceptance borrows an unrelated command event | Proposal and target integrity | Same-workspace event exists for another target/command/proposal | Accept locks proposal+target, executes normal command, requires workspace/target match, causation=submission event, payload proposal ID | Foreign, wrong target/type/command/causation/payload event aborts target/event/proposal atomically; N100-26, N110-37..43 | KBN-100, 110 | Event payload schema drift | Controlled by KBN-105 fixtures |
|
||||
| T19 | Application role updates/deletes audit, approval evidence, checkpoint, or artifact | Nonrepudiation | Broad DB grants or parent cascade exists | INSERT/SELECT-only application roles; RESTRICT parent deletes; archive/cancel normal lifecycle | Role-level UPDATE/DELETE denied; parent delete RESTRICT; digest unchanged; N100-27..31 | KBN-100 | DB superuser can alter state | Break-glass/infra audit residual |
|
||||
| T20 | Break-glass purge is used as routine deletion or erases its own evidence | Retention and incident forensics | Elevated credential available | Separate audited retention procedure, bounded scope, reason, pre/post evidence, authority separation | Normal role denied; expired/missing approval denied; purge cannot delete its authorizing audit package; N115-01, N230-02/03 | KBN-115, 230 | Privileged DBA compromise | Accepted operational residual |
|
||||
| T21 | PostgreSQL unavailable or partitioned | Canonical state | Public health/Valkey remains live while transaction probe fails | Fail closed; no alternate writer/hidden queue; 503 only for proven not-applied; transport uncertainty remains unknown | Fault injection proves DB rows/outbox/files/Valkey unchanged on deliberate denial; commit-unknown replay; N110-44..48, N140-01 | KBN-110, 140, 230 | Availability loss is intentional | Accepted by Option A |
|
||||
| T22 | Valkey unavailable, duplicated, stale, or partitioned | Scheduling notifications | Queue wake is treated as truth or publication fails | Valkey derived/expendable; transactional outbox in PG; idempotent publisher; recovery from PG | Commit with Valkey down leaves pending outbox; replay publishes once logically; stale wake reloads PG; N110-49, N140-02, N230-04..06 | KBN-110, 210, 230 | Duplicate at-least-once delivery | Consumers must be idempotent |
|
||||
| T23 | Coordinator restarts between assignment, lease, checkpoint, or outbox steps | Durable orchestration truth | Process-local cache is treated as authority | PostgreSQL stores assignments, execution state, leases, fences, checkpoints, events, outbox; `recoverFromPostgres` | Restart at every transaction boundary reconstructs identical active/expired/pending sets without Valkey/files; N210-32..36, N230-07 | KBN-210, 230 | Recovery latency | Controlled after evidence |
|
||||
| T24 | Dependency cycle or concurrent reciprocal edge | Readiness and dispatch safety | Two transactions each see an acyclic graph before inserting | Unique directed edge; no self-edge; serialized recursive cycle check; readiness evaluates all blockers | Self/duplicate/cycle and concurrent A→B/B→A tests; all predecessor property test; N100-32..35, N200-01/02 | KBN-100, 200, 230 | Very large DAG performance | Bounded operational residual |
|
||||
| T25 | Parent-task cycle or project-incongruent relation | Planning hierarchy | Valid same-workspace IDs are arranged into an invalid tree | Project-congruent composites; serialized parent-cycle/orphan validation required by REQ-PLAN-001 | Self/indirect parent cycle, orphan, and cross-project mission/milestone/parent tests; N100-06..10 | KBN-100, 110 | Cycle validation is service/transaction enforced | Controlled after evidence |
|
||||
| T26 | Concurrent update, duplicate retry, or idempotency payload drift | Aggregate consistency | Two clients use same version/key with different payloads | Expected-version check; semantic event and outbox in same transaction; key returns prior immutable result only for identical command | One update wins; stale gets 409; duplicate identical returns prior; payload drift rejected; N110-50..54, N140-03 | KBN-105, 110, 140 | Long-lived clients face visible conflicts | Intentional user-visible residual |
|
||||
| T27 | State/event/outbox partial commit | Audit and notification consistency | Separate transactions or exception after state write | One PostgreSQL transaction for state+semantic event+outbox | Failure injected after each insert rolls all three back; success revisions align; N110-55..58 | KBN-110, 140 | Outbox publication remains asynchronous | Controlled after evidence |
|
||||
| T28 | Malicious/incorrect importer injects foreign workspace data or dispatchable work | Migration integrity | Source keys collide, lineage is absent, or importer has direct DB authority | Immutable source snapshots/checksums; one-way Gateway/migration-only port; workspace-safe idempotent modes; shadow records cannot dispatch | Foreign/malformed/duplicate/partial-resume/lineage checksum and no-dispatch tests; N300-01..08 | KBN-300, 330 | Source data may be semantically ambiguous | Quarantine and owner sign-off |
|
||||
| T29 | Cutover leaves legacy writer or forward/reverse sync active | Sole-writer invariant | Credentials/processes survive switch or rollback is improvised | Writer inventory, freeze, final delta, Gateway switch, credential shutdown, no dual write; rollback authority changes after first DB mutation | Process/credential inventory; concurrent-writer assertion; before/after-mutation rollback rehearsal; N320-01..06, N330-01 | KBN-320, 330, 340 | Missed external automation | Owner-gated residual |
|
||||
| T30 | Generated `TASKS.md`/`mission.json` is edited or parsed into DB | Canonical state | Current-main parser/writer remains reachable or file watcher imports changes | Generated non-authoritative header/IDs/time/revision; no production importer; regenerate/overwrite only | Static import search, tamper/regeneration, read-only permission, source-revision parity; N120-03..07, N140-04 | KBN-120, 140 | Humans may mistake snapshots for live data | Header and docs mitigate |
|
||||
| T31 | N-1 compatibility copies legacy ambiguity into canonical authority | Data integrity | Nullable/global/current-main fields are guessed during backfill | Nullable-first expand; deterministic mapping or quarantine; checksums; no new-only status before switch; legacy fields retained | Production-shape, ambiguous owner/assignee, status shadow, JSON/config/digest, rollback tests; N100-36..44 | KBN-100 | Quarantined records require human decision | Controlled by signed reconciliation |
|
||||
| T32 | Recovery posture claims durability not provided by mechanisms | Availability and audit retention | Shape-only validation or optimistic RPO is accepted | Normative validator; WAL/PITR/RPO/storage/high-assurance constraints; mechanism and restore evidence | Unknown/impossible/weakened configuration plus actual mechanism/restore tests; N115-02..08 | KBN-115 | Backup operator or storage compromise | Separate failure domain residual |
|
||||
| T33 | rc.3 frozen DDL could not create mission-scoped evidence/approval FKs | Tenant/evidence relational integrity | KBN-100 generated DDL from the rc.3 contract without an exact composite candidate key | rc.4 adds non-partial `missions_workspace_id_uidx(workspace_id,id)` before both dependent FKs while retaining global and project-congruent keys | KBN-100 must execute N100-45..50: exact-key reconciliation, candidate-before-FKs, duplicate feasibility, empty/prod/N-1/rollback, and both-child foreign-workspace negatives | KBN-100 after PR/CI/#753 release | Runtime DDL remains unimplemented and must prove the frozen order | **Resolved by rc.4 + independent APPROVE; implementation evidence remains required** |
|
||||
|
||||
## 5. Constraint-impact matrix
|
||||
|
||||
| Impact ID | Required invariant | Frozen schema impact | API/transaction impact | Required evidence | Owner | Status |
|
||||
| --------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- | --------------------------------------------------------------------------------- |
|
||||
| CI-01 | Hard workspace tenancy and no oracle | `workspace_id`, workspace-aware unique/FKs on all canonical rows | Server-derived workspace; uniform denial on all surfaces | N100-01..14; N110-01..09; N130-01 | KBN-100/105/110/130 | Resolved by frozen controls |
|
||||
| CI-02 | Active user membership | Membership row plus unique `(workspace_id,user_id)`; active state retained | Recheck active membership in same authoritative transaction | N100-03; N110-06/07 | KBN-100/110 | Resolved; not FK-only |
|
||||
| CI-03 | Service identity least privilege/revocation | Agent/session workspace, lifecycle, state, roles, capabilities | Token maps to exact agent/session; command-family allowlist; DB recheck; no admin/raw DB fallback | N105-01; N110-10..13; N210-01/02 | KBN-105/110/210 | Resolved at auth/API layer |
|
||||
| CI-04 | Project-congruent hierarchy | Composite project/mission/milestone/parent/current-milestone relations | Lock/serialized parent-cycle and orphan validation | N100-06..10 | KBN-100/110 | Resolved; cycle behavior required |
|
||||
| CI-05 | Health-proof authority | Internal branded proof has transaction/time/policy fields | Probe and revalidate on same PG transaction; no public field | N105-02/03; N110-14..27 | KBN-105/110 | Resolved by frozen controls |
|
||||
| CI-06 | Assignment/approval identity | Exactly-one principal/proposer, exact agent/session assignment, relational approval | Reload+lock all IDs; compare version/target/state/expiry/policy/decision | N100-15..18; N210-03..19 | KBN-100/210 | Resolved by frozen controls |
|
||||
| CI-07 | Monotonic bigint fencing | Durable bigint counter, exact lease/fence keys, one active lease | Atomic increment/RETURNING; decimal-string DTO; reject every stale worker command | N100-19..23; N210-20..31 | KBN-100/105/210 | Resolved by frozen controls |
|
||||
| CI-08 | Proposal event chain | Both workspace-aware event FKs; event table created first | Exact submission/acceptance semantic checks in one transaction | N100-24..26; N110-28..43 | KBN-100/110 | Resolved; semantic checks not FK-only |
|
||||
| CI-09 | Immutable audit/evidence retention | RESTRICT parents; INSERT/SELECT-only immutable tables | Archive/cancel normal flow; separately authorized purge | N100-27..31; N115-01; N230-02/03 | KBN-100/115/230 | Resolved by frozen controls |
|
||||
| CI-10 | DB/Valkey/outbox/restart semantics | PG outbox and durable orchestration rows | Fail closed; same-key uncertainty retry; Valkey reloads PG; restart from PG | N110-44..49; N140-01/02; N230-04..07 | KBN-110/210/230 | Resolved by frozen controls |
|
||||
| CI-11 | DAG/race/idempotency/version | Unique edge; self check; event idempotency; aggregate versions | Serialized recursive cycle check; payload binding; expected-version conflict | N100-32..35; N110-50..58; N200-01/02 | KBN-100/110/200 | Resolved by frozen controls |
|
||||
| CI-12 | Import/cutover trust boundary | Lineage/artifact/event fields; shadow state cannot dispatch | One-way scoped importer, freeze, no direct DB/file authority, no dual writer | N300-01..08; N320-01..06 | KBN-300/320/330 | Resolved by frozen controls |
|
||||
| CI-13 | Generated-file no-import | No canonical file schema/import contract | Projection-only package; static reachability check removes current parser from production Kanban paths | N120-03..07; N140-04 | KBN-120/140 | Resolved by frozen controls |
|
||||
| CI-14 | Mission-scoped artifact and approval FKs | rc.4 adds non-partial `missions_workspace_id_uidx(workspace_id,id)` and retains global/project-congruent keys | KBN-100 must emit the candidate before both exact RESTRICT FKs and preserve N-1/rollback order | N100-45..50: exact reconciliation, duplicate feasibility, empty/prod/N-1/rollback, and separate artifact/approval foreign-workspace negatives | KBN-100 after PR/CI/#753 release | **Resolved by rc.4 and independent APPROVE; future executable evidence required** |
|
||||
|
||||
## 6. Exact future negative-test catalog
|
||||
|
||||
These names are normative evidence identifiers for future slices. Equivalent test-file names are acceptable only if traceability retains these IDs and expected outcomes.
|
||||
|
||||
### KBN-100 — schema and migration
|
||||
|
||||
- **N100-01** reject every canonical child row whose `workspace_id` differs from its parent.
|
||||
- **N100-02** reject foreign-workspace link, artifact, proposal target, dependency, assignment, lease, checkpoint, approval, and event relationships.
|
||||
- **N100-03** reject an inactive/revoked member as accountable owner, proposer, decision actor, archive actor, or user principal in the authoritative command transaction.
|
||||
- **N100-04** reject a team/project relation crossing workspaces.
|
||||
- **N100-05** reject a team authorization path when the user lacks active membership in the team's workspace.
|
||||
- **N100-06** reject task→mission project mismatch.
|
||||
- **N100-07** reject task→milestone and project→current-milestone project mismatch.
|
||||
- **N100-08** reject task→parent project mismatch and self-parent.
|
||||
- **N100-09** reject indirect parent cycles under concurrent transactions.
|
||||
- **N100-10** reject mission→milestone project mismatch/orphan.
|
||||
- **N100-11** reject checkpoint artifact from another workspace.
|
||||
- **N100-12** reject checkpoint artifact owned by another same-workspace task/mission unless an explicitly frozen evidence rule permits it.
|
||||
- **N100-13** reject approval evidence from another workspace.
|
||||
- **N100-14** reject same-workspace approval evidence unrelated to the approval target.
|
||||
- **N100-15** reject zero/multiple assignment principals and zero/multiple proposers.
|
||||
- **N100-16** reject target session without its exact target agent.
|
||||
- **N100-17** reject assignment task/agent/session crossing workspaces.
|
||||
- **N100-18** reject non-positive task version and expired assignment acquisition.
|
||||
- **N100-19** concurrent lease insert permits one active lease and returns one winner.
|
||||
- **N100-20** successive leases return strictly increasing bigint fences.
|
||||
- **N100-21** reject checkpoint with another task, lease, or fence.
|
||||
- **N100-22** reject duplicate/non-monotonic checkpoint sequence.
|
||||
- **N100-23** reject evidence join for a mismatched checkpoint/task.
|
||||
- **N100-24** proposal insert without exact submission event fails atomically.
|
||||
- **N100-25** foreign/wrong-type/wrong-proposal submission event fails atomically.
|
||||
- **N100-26** foreign/wrong-target/unrelated acceptance event fails atomically.
|
||||
- **N100-27** application role cannot UPDATE/DELETE `task_events`.
|
||||
- **N100-28** application role cannot UPDATE/DELETE checkpoints/artifacts/evidence joins.
|
||||
- **N100-29** parent hard delete is RESTRICTed while audit/evidence children exist.
|
||||
- **N100-30** archive does not alter canonical lifecycle status.
|
||||
- **N100-31** purge without break-glass authority/evidence is denied.
|
||||
- **N100-32** reject dependency self-edge and duplicate directed pair regardless of type.
|
||||
- **N100-33** reject direct and indirect dependency cycles.
|
||||
- **N100-34** concurrent reciprocal dependency inserts cannot both commit.
|
||||
- **N100-35** readiness remains false until every blocking predecessor and completion condition passes.
|
||||
- **N100-36** empty DB migration succeeds after the contract amendment.
|
||||
- **N100-37** production-shape expand retains all legacy declarations.
|
||||
- **N100-38** crash/resume backfill is idempotent and checksum-stable.
|
||||
- **N100-39** ambiguous workspace/owner/assignee is quarantined, never guessed.
|
||||
- **N100-40** no `ready`/`in_review` status is emitted to N-1 readers before switch.
|
||||
- **N100-41** `mission_tasks.status` cannot remain a write source.
|
||||
- **N100-42** tags/assignee/date/mission JSON/config/description/agent fields reconcile without loss.
|
||||
- **N100-43** claimed fleet backlog rows are quarantined and imported rows cannot dispatch.
|
||||
- **N100-44** pre-switch rollback works while post-first-mutation rollback requires freeze/reconciliation.
|
||||
- **N100-45** reconcile both exact child FK column lists to the rc.4 `(workspace_id,id)` mission candidate while retaining the global `id` primary key and `(workspace_id,project_id,id)` key.
|
||||
- **N100-46** empty-DB migration creates `missions_workspace_id_uidx` before `artifacts_workspace_mission_fk` and `approval_decisions_workspace_mission_fk`.
|
||||
- **N100-47** production-shape preflight finds no duplicate `(workspace_id,id)` groups, preserves global `id` uniqueness, and applies the candidate before both dependent FKs.
|
||||
- **N100-48** N-1 startup/read/write remains unchanged; pre-switch rollback drops both dependents before the candidate and preserves the global/project-congruent keys.
|
||||
- **N100-49** artifact insert using a valid mission ID paired with a foreign workspace fails before commit.
|
||||
- **N100-50** approval-decision insert using a valid mission ID paired with a foreign workspace fails before commit.
|
||||
|
||||
### KBN-105/KBN-110/KBN-120/KBN-130/KBN-140 — API and P1
|
||||
|
||||
- **N105-01** every route has an explicit user/service command-family policy; user/admin tokens cannot call service-only Coordinator mutations.
|
||||
- **N105-02** public DTO validation rejects `writeProof`, internal context, body `workspaceId`, and caller-asserted health.
|
||||
- **N105-03** fixture exhaustiveness prevents 503, 502/504/timeout, and 409 cross-mapping.
|
||||
- **N105-04** approval DTO accepts an ID and decision command only, never approval proof-by-value.
|
||||
- **N105-05** all fence fields accept/emit decimal strings and reject JSON numbers.
|
||||
- **N110-01** listing with a foreign `workspaceId` or foreign filter ID follows the frozen no-oracle denial and returns no rows/counts/cursors.
|
||||
- **N110-02** get by foreign or nonexistent aggregate ID has the same frozen denial shape and no foreign metadata.
|
||||
- **N110-03** create/update/archive with a foreign owner, parent, project, mission, milestone, tag, or target ID is denied before mutation.
|
||||
- **N110-04** dependency/proposal commands with foreign target IDs are denied with unchanged state/event/outbox counts.
|
||||
- **N110-05** REST, MCP, WebSocket, and internal Coordinator paths produce equivalent no-oracle behavior for the same foreign ID.
|
||||
- **N110-06** a revoked/inactive owner is denied even with a still-valid Better Auth session.
|
||||
- **N110-07** stale membership/team cache cannot authorize a proposer, decision actor, archive actor, or principal after revocation.
|
||||
- **N110-08** a team ID from another workspace cannot authorize or own the command target.
|
||||
- **N110-09** same-workspace but wrong-project mission/milestone/parent IDs are denied inside the transaction.
|
||||
- **N110-10** an expired service token is denied before repository access.
|
||||
- **N110-11** an audience- or workspace-mismatched service token is denied without an existence oracle.
|
||||
- **N110-12** an over-scoped service token cannot call a command family absent from its role/capability allowlist.
|
||||
- **N110-13** disabled agent or ended session revokes service-token command authority immediately on PostgreSQL recheck.
|
||||
- **N110-14** contradictory public health state/boolean combinations fail validation.
|
||||
- **N110-15** Valkey-only liveness cannot mint or substitute a PostgreSQL write proof.
|
||||
- **N110-16** caller-forged public `healthy` cannot enter internal mutation context.
|
||||
- **N110-17** public REST/MCP/CLI bodies containing health/proof fields are rejected.
|
||||
- **N110-18** an expired internal proof produces no state/event/outbox write.
|
||||
- **N110-19** a future-dated or not-yet-valid proof produces no write.
|
||||
- **N110-20** a policy-revision-mismatched proof produces no write.
|
||||
- **N110-21** a proof minted on another transaction/connection produces no write.
|
||||
- **N110-22** a proof that expires before the final pre-mutation check produces no write.
|
||||
- **N110-23** deliberate read-only/write-unavailable denial maps only to authoritative 503/not-applied/non-retryable.
|
||||
- **N110-24** timeout before commit maps to transport-unknown and permits only same-key retry.
|
||||
- **N110-25** timeout after commit maps to transport-unknown and same-key retry returns the committed canonical result once.
|
||||
- **N110-26** expected-version mismatch maps only to 409/not-applied/non-retryable.
|
||||
- **N110-27** recovery replay with a changed idempotency key cannot masquerade as the original uncertain request.
|
||||
- **N110-28** pending proposal cannot alter target fields/status/rank/version.
|
||||
- **N110-29** rejected proposal cannot affect readiness, dependencies, or gates.
|
||||
- **N110-30** pending/rejected proposal cannot create an assignment or lease.
|
||||
- **N110-31** direct proposal-row state manipulation cannot bypass normal command execution.
|
||||
- **N110-32** proposal submission without a submission event rolls back fully.
|
||||
- **N110-33** foreign-workspace submission event rolls back fully.
|
||||
- **N110-34** wrong aggregate/event type submission event rolls back fully.
|
||||
- **N110-35** same-workspace event for another proposal rolls back fully.
|
||||
- **N110-36** submission event with wrong previous/new version semantics rolls back fully.
|
||||
- **N110-37** foreign-workspace acceptance event rolls back proposal, target, event, and outbox.
|
||||
- **N110-38** same-workspace event for another target aggregate rolls back acceptance.
|
||||
- **N110-39** event from an unrelated normal command rolls back acceptance.
|
||||
- **N110-40** event caused by a different submission event rolls back acceptance.
|
||||
- **N110-41** event whose payload lacks or changes `changeProposalId` rolls back acceptance.
|
||||
- **N110-42** event for another proposal with the same target/command rolls back acceptance.
|
||||
- **N110-43** missing accepted-command event after target handling rolls back the entire transaction.
|
||||
- **N110-44** read-only-degraded denial changes no DB row/outbox/file/Valkey/provider state.
|
||||
- **N110-45** write-unavailable denial changes no DB row/outbox/file/Valkey/provider state.
|
||||
- **N110-46** PostgreSQL disconnect cannot redirect a command to any fallback writer.
|
||||
- **N110-47** commit uncertainty remains `unknown` and never becomes a fabricated 503/not-applied result.
|
||||
- **N110-48** same-key replay after recovery returns one canonical result with no duplicate event/outbox row.
|
||||
- **N110-49** Valkey publication failure leaves committed PG outbox pending and replayable.
|
||||
- **N110-50** two same-version updates produce one winner and one visible 409 loser.
|
||||
- **N110-51** identical duplicate key+payload returns the prior immutable result without another event/outbox row.
|
||||
- **N110-52** same key with payload/command drift is rejected as an idempotency conflict.
|
||||
- **N110-53** the same key in another workspace cannot reveal or reuse the first workspace's result.
|
||||
- **N110-54** stale reconnect/update cannot silently overwrite a newer aggregate revision.
|
||||
- **N110-55** failure after state write but before semantic event rolls back state.
|
||||
- **N110-56** failure after semantic event but before outbox rolls back state and event.
|
||||
- **N110-57** failure after outbox insert but before commit rolls back state, event, and outbox.
|
||||
- **N110-58** success commits matching aggregate/event/outbox revisions and correlation/causation.
|
||||
- **N120-01** CLI never retries an authoritative 503 deliberate denial.
|
||||
- **N120-02** CLI retries only transport-unknown outcomes and preserves the exact idempotency key.
|
||||
- **N120-03** generated projection header contains non-authoritative warning, workspace/project IDs, generated time, and source revision.
|
||||
- **N120-04** projection revision and records match the API snapshot revision exactly.
|
||||
- **N120-05** hand-tampering is overwritten or rejected by regeneration and never mutates PostgreSQL.
|
||||
- **N120-06** static/runtime reachability finds no parser/import path from `TASKS.md`, `mission.json`, or another export.
|
||||
- **N120-07** projection writer has no domain mutation/raw SQL/Valkey authority.
|
||||
- **N130-01** UI foreign/no-access/not-found state follows the frozen no-oracle response and renders no stale foreign data.
|
||||
- **N140-01** real-Gateway DB fault journey proves fail-closed no-fallback behavior.
|
||||
- **N140-02** real-Gateway Valkey-loss journey proves pending outbox replay.
|
||||
- **N140-03** real-Gateway concurrent update/retry journey proves version and idempotency semantics.
|
||||
- **N140-04** generated-file tamper journey proves projection parity and no import.
|
||||
|
||||
### KBN-115/KBN-200/KBN-210/KBN-230 — recovery and coordination
|
||||
|
||||
- **N115-01** retention purge without current break-glass authority, reason, immutable evidence, or bounded scope is denied and audited.
|
||||
- **N115-02** recovery posture with an unknown top-level or storage field is rejected.
|
||||
- **N115-03** PITR retention without WAL archival is rejected.
|
||||
- **N115-04** WAL archival with zero PITR retention is rejected.
|
||||
- **N115-05** claimed RPO better than the configured backup/WAL mechanism is rejected.
|
||||
- **N115-06** unencrypted, optional, or same-failure-domain storage is rejected.
|
||||
- **N115-07** weakened high-assurance values are rejected.
|
||||
- **N115-08** shape-only validation cannot pass without normative mechanism and restore evidence.
|
||||
- **N200-01** cyclic/incomplete dependency snapshots never become eligible.
|
||||
- **N200-02** identical immutable snapshot+policy+time returns identical ordering and explanation with no I/O/model import.
|
||||
- **N210-01** disabled agent cannot claim, ack, heartbeat, checkpoint, or submit review.
|
||||
- **N210-02** ended/offline/mismatched session cannot claim, ack, heartbeat, checkpoint, or submit review.
|
||||
- **N210-03** foreign-workspace task is rejected after lock/reload without an oracle.
|
||||
- **N210-04** stale task version is rejected before fence increment.
|
||||
- **N210-05** assignment target agent mismatch is rejected.
|
||||
- **N210-06** target session mismatch is rejected.
|
||||
- **N210-07** expired assignment is rejected.
|
||||
- **N210-08** assignment in rejected/released/expired/superseded/leased-invalid state is rejected.
|
||||
- **N210-09** missing approval is rejected.
|
||||
- **N210-10** rejected/escalated/requested approval is rejected as approval authority.
|
||||
- **N210-11** stale policy-revision approval is rejected.
|
||||
- **N210-12** foreign-workspace approval is rejected without an oracle.
|
||||
- **N210-13** approval for another assignment is rejected.
|
||||
- **N210-14** author self-approval/review is rejected when independence is required.
|
||||
- **N210-15** foreign-workspace artifact evidence is rejected.
|
||||
- **N210-16** same-workspace artifact unrelated to the assignment/task/gate is rejected.
|
||||
- **N210-17** concurrent policy revocation versus acquire cannot produce a lease under the revoked revision.
|
||||
- **N210-18** concurrent assignment expiry versus acquire cannot produce a lease after expiry.
|
||||
- **N210-19** concurrent session end versus acquire cannot produce a lease for the ended session.
|
||||
- **N210-20** lower fencing token is rejected without writes.
|
||||
- **N210-21** token from an older lease is rejected without writes.
|
||||
- **N210-22** token paired with another task is rejected without writes.
|
||||
- **N210-23** token paired with another session is rejected without writes.
|
||||
- **N210-24** token on an expired/revoked/released lease is rejected without writes.
|
||||
- **N210-25** fences above JavaScript safe integer round-trip exactly as decimal strings.
|
||||
- **N210-26** lease task does not match assignment task and is rejected.
|
||||
- **N210-27** lease agent/session does not match assignment target and is rejected.
|
||||
- **N210-28** checkpoint task does not match lease task and is rejected.
|
||||
- **N210-29** checkpoint fence does not match exact lease fence and is rejected.
|
||||
- **N210-30** checkpoint sequence duplicate/regression is rejected.
|
||||
- **N210-31** checkpoint artifact does not match workspace/task/evidence semantics and is rejected.
|
||||
- **N210-32** restart after assignment persistence reconstructs the pending assignment.
|
||||
- **N210-33** restart after lease commit reconstructs exact active lease and fence.
|
||||
- **N210-34** restart after checkpoint commit reconstructs checkpoint/recovery state.
|
||||
- **N210-35** restart during expiry/retry/quarantine reconstructs durable disposition and eligibility.
|
||||
- **N210-36** restart with pending outbox reconstructs publication work without Valkey/files.
|
||||
- **N230-01** author=self-review and missing mandatory SecReview cannot certify or complete.
|
||||
- **N230-02** normal application role cannot execute retention purge.
|
||||
- **N230-03** break-glass purge cannot delete or alter its own authorization/evidence chain.
|
||||
- **N230-04** Valkey down leaves canonical work in PostgreSQL/outbox.
|
||||
- **N230-05** duplicate wake produces one logical effect after PostgreSQL reload/idempotency.
|
||||
- **N230-06** stale wake cannot revive an expired/revoked assignment or lease.
|
||||
- **N230-07** restart with no Valkey/files reconstructs leases/retry/quarantine/outbox exactly.
|
||||
|
||||
### KBN-300/KBN-320/KBN-330/KBN-340 — migration and cutover
|
||||
|
||||
- **N300-01** source record targeting another workspace is denied/quarantined without an oracle.
|
||||
- **N300-02** malformed source record is rejected with attributable reject evidence.
|
||||
- **N300-03** duplicate source system/key/batch replay is idempotent.
|
||||
- **N300-04** source snapshot/checksum drift aborts apply/verify.
|
||||
- **N300-05** partial import resumes from durable lineage without duplicating state/events.
|
||||
- **N300-06** imported shadow record cannot become ready, assigned, or leased automatically.
|
||||
- **N300-07** missing source key/file/checksum/batch lineage prevents apply/sign-off.
|
||||
- **N300-08** importer cannot use direct DB, generated file, Valkey, or provider issue as canonical write authority.
|
||||
- **N320-01** cutover without a verified write freeze fails safe.
|
||||
- **N320-02** active legacy writer process or credential blocks cutover.
|
||||
- **N320-03** reverse and forward synchronization cannot run concurrently.
|
||||
- **N320-04** failed final delta/reconciliation blocks client switch.
|
||||
- **N320-05** rollback before first canonical DB mutation may switch authority back only after freeze assertion.
|
||||
- **N320-06** rollback after first canonical mutation requires freeze, DB-delta export/reconciliation, and owner decision.
|
||||
- **N330-01** rehearsal cannot sign off while counts/checksums/exceptions/writer inventory differ.
|
||||
- **N340-01** cutover cannot proceed without owner authorization, terminal evidence, scoped identities, and zero active legacy writers.
|
||||
|
||||
## 7. Requirements traceability
|
||||
|
||||
| Requirement | Threats/impacts | Planned evidence |
|
||||
| ---------------- | ---------------------------- | --------------------------------------------------------------- |
|
||||
| REQ-SOT-001 | T16, T21, T22, T27, T29, T30 | N110-28..31, N110-44..49, N110-55..58, N120-03..07, N320-01..06 |
|
||||
| REQ-SOT-002 | T07, T08, T09, T21 | N105-02/03, N110-14..27, N110-44..48 |
|
||||
| REQ-SOT-003 | T30 | N120-03..07, N140-04 |
|
||||
| REQ-SOT-004 | T16..18 | N100-24..26, N110-28..43 |
|
||||
| REQ-TEN-001 | T01..05, T15, T33 | N100-01..14, N100-45..50, N110-01..09, N210-15/16 |
|
||||
| REQ-ID-001 | T02, T03, T06, T10..12 | N105-01, N110-06..13, N210-01..19 |
|
||||
| REQ-PLAN-001 | T04, T25 | N100-06..10 |
|
||||
| REQ-TASK-001 | T13, T26, T31 | N100-20, N100-37..42, N110-50..54 |
|
||||
| REQ-TASK-002 | T16, T24 | N110-28..31, N100-35, N200-01 |
|
||||
| REQ-DEP-001 | T24 | N100-32..35, N200-01 |
|
||||
| REQ-ASN-001 | T10..12 | N100-15..18, N210-03..19 |
|
||||
| REQ-AUD-001 | T17..20, T22, T27 | N100-24..31, N110-32..43, N110-49, N110-55..58 |
|
||||
| REQ-API-001 | T01, T06..18, T26 | N105-01..05 plus KBN-110 catalog |
|
||||
| REQ-UI-002/003 | T01, T15, T26 | N130-01 and real-Gateway KBN-140 journeys |
|
||||
| REQ-COORD-001 | T22..24 | N200-01/02, N210-32..36 |
|
||||
| REQ-COORD-002 | T10..12, T16 | N210-03..19, N110-28..31 |
|
||||
| REQ-COORD-003 | T13..15, T23 | N100-19..23, N210-20..36 |
|
||||
| REQ-COORD-004 | T23, T26 | N210-32..36, N230-07 |
|
||||
| REQ-GATE-001/002 | T11, T19, T20 | N210-09..14, N230-01..03 |
|
||||
| REQ-REC-001 | T20, T32 | N115-01..08 |
|
||||
| REQ-MIG-001/002 | T28, T29, T31 | N100-37..44, N300-01..08, N320-01..06, N330-01, N340-01 |
|
||||
|
||||
REQ-UI-001 and REQ-UI-004 are downstream functional/accessibility requirements rather than schema-threat controls; they remain owned by KBN-130/KBN-140. Their security-relevant tenancy, conflict, and stale-reconnect portions are covered above.
|
||||
|
||||
## 8. Issue #753 acceptance mapping
|
||||
|
||||
| Issue requirement/criterion | Evidence in this document | Result |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------- | ---------------------------------------------- |
|
||||
| Cross-workspace owners, principals, evidence, project hierarchy | T01–T05, T15, T25; CI-01–04 | Mapped |
|
||||
| Active membership and service-token boundaries | Authorization matrix; T02, T03, T06; CI-02/03 | Mapped |
|
||||
| Stale/forged health and transaction-local proof | T07–T09, T21; CI-05 | Mapped |
|
||||
| Assignment/approval forgery and monotonic fencing | T10–T15; CI-06/07 | Mapped |
|
||||
| Change-proposal abuse and event binding | T16–T18; CI-08 | Mapped |
|
||||
| Immutable audit and break-glass | T19/T20; CI-09 | Mapped |
|
||||
| PostgreSQL/Valkey failures | T21–T23; CI-10 | Mapped |
|
||||
| Dependency/idempotency/version races | T24–T27; CI-11 | Mapped |
|
||||
| Import/cutover and generated-file boundary | T28–T31; CI-12/13 | Mapped |
|
||||
| Every schema/API/test impact explicit | Constraint matrix and negative-test catalog | Mapped |
|
||||
| No unresolved schema impact | CI-14; rc.4 resolved-impact record | **PASS — none unresolved** |
|
||||
| Independent SecReview | Homelab non-author exact commit/tree/content review | **PASS / APPROVE** |
|
||||
| PR merge, terminal-green main CI, and #753 closure | Orchestrator-owned post-worker gates | Pending; KBN-100 remains held until completion |
|
||||
|
||||
## 9. UNRESOLVED SCHEMA IMPACTS
|
||||
|
||||
none
|
||||
|
||||
### Resolved-impact record — KBN010-SI-001
|
||||
|
||||
- **Historical detection:** rc.3 lacked an exact `(workspace_id,id)` candidate key for the artifact and approval-decision mission FKs. This document's original BLOCKED verdict was correct and remains preserved in §1 and T33.
|
||||
- **Resolution:** rc.4 adds non-partial `missions_workspace_id_uidx(workspace_id,id)` before both exact dependent FKs while retaining the global primary key and project-congruent key.
|
||||
- **Reviewed object:** commit `3f6a3387b419eb99453ee10dd25ba888faaab0b5`, tree `7ebab8fa530a7180036928cea9527f808548aa14`.
|
||||
- **Corroborating identities:** full-index SHA-256 `6b40a76265c4f3e6d1d30a7f262a2dd16e0d51997e99c146b59f527e6524cd42`; stable patch-id `058cf98026fcd1043703c866aee047c8bb144740`.
|
||||
- **Independent verdict:** Homelab non-author schema/security review **APPROVE**. It confirmed PostgreSQL candidate/FK validity, unchanged tenant and polymorphic exactly-one-target safety, RESTRICT/no-cascade semantics, N-1/rollback validity, and no shared table/index/FK/identity/fence authority collision with #757.
|
||||
- **Digest interpretation:** a command-rendered patch digest varied with rendering command/options and is non-authoritative. Git commit + tree + exact file content are canonical; stable full-index SHA-256 and stable patch-id corroborate that identity.
|
||||
- **Residual implementation obligations:** KBN-100 must create the candidate before both dependent FKs; prove production-shape duplicate feasibility without weakening global uniqueness; pass empty/prod/N-1/rollback tests; reconcile both exact FK targets; and separately reject foreign-workspace mission references for artifacts and approval decisions (N100-45..50).
|
||||
- **Implementation status:** no runtime schema, migration, API, or deployment implementation is claimed by this gate disposition.
|
||||
|
||||
## 10. Residual risk and handoff
|
||||
|
||||
- Active membership, polymorphic targets, same-task evidence semantics, parent/DAG cycle checks, token scope, and no-oracle behavior depend on authoritative transaction code and must not be treated as FK-only guarantees.
|
||||
- DB superuser and break-glass compromise cannot be eliminated by application constraints; separation of duties, immutable external backup/audit evidence, drills, and monitoring remain required.
|
||||
- PostgreSQL unavailability intentionally sacrifices writes for integrity. Transport-unknown outcomes remain safe only when clients preserve the exact idempotency key.
|
||||
- Imported ambiguous records remain quarantined until owner sign-off; no automated mapping may convert ambiguity into authority.
|
||||
- SI-001 is resolved at frozen contract/design-review level only. KBN-100 still owes N100-45..50 executable migration evidence.
|
||||
|
||||
**Handoff status:** KBN-010 **PASS / GO** at rc.4. KBN-100 remains held until this PR squash-merges, terminal-green CI completes on `main`, and issue #753 closes; the orchestrator owns those remaining gates.
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,377 @@
|
||||
# KBN-101 — B1/B2 Envelope A (v6, FINAL) — Declarative Sink-RBAC + Per-Role Credential/Connection-Selection + RLS Write-Source (INSERT tenant-bound, single-compound-or-RESTRICTIVE composition) + Sink-Resident User-Override + Read/USING Enforcement
|
||||
|
||||
**Ratification status:** RATIFIED — part of the frozen SSOT as of this PR (KBN-101 Envelope A, landed **FORM A — apply-in-place**). This is the converged **v6** envelope, ratified as the authoritative record of the **rc.20** contract amendment now inlined into [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md) §4/§10, [`SHARED-CONTRACT.md`](./SHARED-CONTRACT.md) (`### 1.0.0-rc.20`), and [`MISSION-MANIFEST.md`](./MISSION-MANIFEST.md) §10.
|
||||
|
||||
**Ratification lineage (authoritative):** Jason B1 ruling (2026-07-20T23:08Z) + Mos coordinator **OPTION A** (home the layer) + **Mos Q1** ruling (TIER-LEVEL) + **Mos Q2** ruling (RLS `WITH CHECK` AUTHORIZED, two mandatory conditions — `ENABLE`/`FORCE ROW LEVEL SECURITY` as the terminal migration step, and non-owner rung roles) + **Gate A** (`a11a1e2c`) + **Gate B** (`a6aac2cd`).
|
||||
|
||||
> **Ratification note.** The sections below are the converged v6 envelope text, retained **VERBATIM** for traceability. Their design-time framing — e.g. "**Scope:** DESIGN / CONTRACT-AMENDMENT only. No implementation code, no PRs, no SSOT edits." and "**Base:** … this doc modifies nothing there." — describes the envelope as originally authored against base `b0d78d86`. As of this PR that rc.20 amendment is **ratified and applied in-place (FORM A)** to the frozen SSOT; the design-time "no SSOT edits" wording is historical authoring context, not a live constraint on this ratified record. The Ratification lineage above is authoritative.
|
||||
|
||||
---
|
||||
|
||||
**Status:** v6 envelope (design / contract-amendment), FINAL. **v6 = v5 + N1/N2/F-NB4 non-blocking test/lint/cert hardening ONLY; NO design predicate/policy/grant/role/table change.** v5 CONVERGED — BOTH terminal re-gate-4 gates returned GO (Gate A-delta-4 `a4ce075a`; Gate B-delta-4 `a379e0de`), NO HALT; Gate B-delta-4 flagged 3 EXPLICITLY NON-BLOCKING hardening notes "for the trunk-commit record," and v6 folds exactly those 3 as additive test/lint/cert wording on an already-BOTH-GO design: **N1** (strengthen the `pg_policy.polpermissive` lint — group by effective role incl. `PUBLIC`/inheritance; reject an internally-disjunctive single `WITH CHECK`; behavioral positive-controls remain the PRIMARY proof), **N2** (state the option-(b) vacuous-deny guard invariant — ≥1 permissive policy per writable (rung, command)), and **F-NB4** (extend owner non-reachability to SET-ROLE assumability — conjoin the `MEMBER` variant of `pg_has_role`). No design predicate, policy, grant, role, or table is changed. v6 supersedes v5 (`a6e790fd`), which supersedes v4 (`a48ad69c`). Preserves EVERY verified v4 closure (all checks passed on v4 EXCEPT one bounded item) and closes the SINGLE remaining BLOCKING finding raised identically by BOTH re-gate-3 gates (Gate A-delta-3 `aa5cbf73` = CHANGES-NEEDED; Gate B-delta-3 `aa881074` = 1 blocking): the **RLS `WITH CHECK` composition semantics** — v4's construction is right but it justifies the composition with a FALSE PostgreSQL claim ("PostgreSQL requires ALL applicable `WITH CHECK` clauses to pass / they AND") and leaves the _realization_ unspecified, so an implementation could realize the status-pin, the workspace-bind, and the override as SEPARATE PERMISSIVE policies which combine with **OR** (not AND) → a foreign-workspace INSERT satisfying only the status-pin would be admitted (B2-INSERT / B-1 re-open). v5 (a) corrects the semantics, (b) **mandates** the composition be realized as a SINGLE compound `WITH CHECK` OR as `AS RESTRICTIVE` policies and **explicitly forbids** the multi-permissive-intended-to-AND realization, and (c) hardens the red-first negatives to an otherwise-valid row + adds a positive-control. It also folds Gate B-delta-3's NB-4-transitive nit (state owner non-reachability as TRANSITIVE membership). This is a PRECISE, LIGHT revision: every v4 closure (3-rung ladder / zero per-user; F1–F7; GA-1/3/4; B-1/B-2/B-3; B2-INSERT workspace bind itself; N-1..N-5; AD-1/AD-2; NB-1..NB-5; continuous -06 scan + L174 GUC fold; rc.20) carries forward intact. Prior rc.18/rc.19 evidence does NOT carry.
|
||||
**Authoring authority:** Jason B1 ruling (2026-07-20T23:08Z) + Mos coordinator **OPTION A** (home the layer) + **Mos Q1 ruling (TIER-LEVEL)** + **Mos Q2 ruling (RLS `WITH CHECK` AUTHORIZED, two mandatory conditions)**. No new coordinator authority is required for v5: the composition fix is a DDL-realization detail + corrected PostgreSQL-semantics statement + test-wording hardening entirely inside the already-authorized `WITH CHECK`/`FORCE RLS` mechanism (Q2); it introduces no new principal, key, or authority axis. **Both delta-3 gates independently confirmed the per-user/same-tier boundary HOLDS** (the crit-1 defect is a TENANT-ISOLATION correctness risk, NOT a per-user authority axis).
|
||||
**Base:** frozen SSOT at `b0d78d86`, `/src/mosaic-stack/docs/native-kanban-sot/` (READ-ONLY; this doc modifies nothing there).
|
||||
**Scope:** DESIGN / CONTRACT-AMENDMENT only. No implementation code, no PRs, no SSOT edits.
|
||||
|
||||
**HALT self-check result (re-run for v5): NO HALT.** No v5 change forces two SAME-TIER writers to hold DIFFERENT WRITE AUTHORITY. The v5 composition fix pins **how** the already-uniform predicates are physically composed (one compound `WITH CHECK`, or `AS RESTRICTIVE`) — it changes no predicate's _content_ and adds no writer-varying axis; both delta-3 gates confirmed this is a tenant-isolation correctness pin, not a per-user distinction. The B2-INSERT fix ANDs a **shared, per-request workspace predicate** (`workspace_id = current_setting('mosaic.workspace_id', true)::uuid`) into every rung's INSERT `WITH CHECK` — applied **uniformly** to every writer in a tier, exactly like the B-2 read/`USING` predicate; it is tenant isolation, never a per-writer distinction. The B-1 sink-resident User-override remains keyed on **(workspace_id, task_id, tier)** — task-identity + tier ONLY, **NO per-federated-writer key** — so every writer inside a given tier is subject to the _identical_ veto for a given task. The five NB fixes (override-relation no-status enumeration; two claim-scopings; owner-toggle-FORCE containment naming; baseline runtime grants for the two capability roles) introduce **zero** per-writer authority. Per-writer _attribution_ remains an AUDIT concern routed to metadata (`task_events.actor_id`), never a DB credential/role/RLS key. The design remains satisfiable with the **fixed 3-rung role ladder** (User / Orchestrator / others), federated identity → rung at authorization time. The Q1 hard re-open trigger (same-tier / different write-authority) is **not** hit. See §5.
|
||||
|
||||
---
|
||||
|
||||
## 0. What the "raw task-status sink" is (grounding — unchanged from v1/v2)
|
||||
|
||||
- The canonical task-status store is **`tasks.status`** (SHARED-CONTRACT §5.4 L214 "canonical authority"); legacy **`mission_tasks.status`** is frozen read-only / prohibited as a write source (SHARED-CONTRACT §5.4 L219; §5.1 phase 1 L165). `tasks` is a **mutable** relation created by **KBN-100**. Canonical statuses are `backlog | ready | in_progress | blocked | in_review | done | cancelled` (SHARED-CONTRACT §3 L134); initial creation state is `backlog` (§5.4 L214 not-started→backlog).
|
||||
- The append-only event relations `task_events, artifacts, task_checkpoints, task_checkpoint_artifacts, approval_decision_artifacts` receive runtime-only `SELECT, INSERT` with `UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER` **denied** (frozen §4 L178). That producer/consumer precedent — grants applied **after** KBN-100 creates the relations — is the mechanism this envelope extends to the mutable `tasks.status` sink and to the new sink-resident override table (§2.9).
|
||||
- Today there is exactly one runtime login, `mosaic_runtime` (LOGIN INHERIT), sole member of NOLOGIN `mosaic_runtime_capability` (§4 L149-150). The sink therefore has **no writer-authority discrimination** — the root of RC19-B1-01.
|
||||
|
||||
The three B1 findings are writes to this sink; the two B2 findings are the privileged-fd → UID-drop handoff used by the runner/importer (§5 L184/L191). This envelope homes one coherent layer over both, now with the User-override and the SELECT/tenant predicate **resident at the sink**.
|
||||
|
||||
**[NB-2 / NB-3 — precise compromise-resistance scope (Mos Q1 accepted co-resident-pool residual; NO HALT).]** The prior "a compromised Gateway cannot bypass a User deny or read cross-workspace" framing is **overstated** and is corrected here. All three rung credentials are co-resident in one Gateway process (DB authenticates the _credential_, never the federated end-user — the accepted Mos Q1 residual), and the workspace GUC is _app-set_ with no per-tenant DB check. Precisely:
|
||||
|
||||
- **B-1 sink-enforces the User veto against the _Orchestrator-rung path_** — an Orchestrator-pool write to a task carrying an active User deny ERRORS in PostgreSQL regardless of app-layer behavior (a _buggy_ or Orchestrator-path-compromised Gateway cannot skip it). It does **NOT** stop a Gateway compromised badly enough to hold the **User** pool: User = god and may itself clear any veto (User-write-only on the override table). That is the accepted co-resident-pool residual, not a new hole.
|
||||
- **B-2 + B2-INSERT sink-enforce tenant isolation against a _buggy_ Gateway** — an omitted/wrong workspace predicate fails **closed** (reads return zero rows; INSERTs ERROR on the NULL/mismatched GUC). They do **NOT** defend a _compromised_ Gateway that forges `mosaic.workspace_id` to a victim tenant, because the GUC carries no per-tenant DB authentication (accepted Mos Q1 residual). The genuine, real closure B2-INSERT delivers is the **buggy / unbound-INSERT** hole: without it, _any_ rung could write a foreign `workspace_id` on INSERT even with a correct GUC set — that is now impossible.
|
||||
|
||||
Defending against a fully compromised Gateway forging identity/tenant requires per-federated-user DB credentials = the HALT boundary; Mos Q1 explicitly accepts this residual and rules NO HALT. The claims in §2.9 (B-1) and §2.2/§2.4 (B-2/INSERT) are scoped to match.
|
||||
|
||||
---
|
||||
|
||||
## 1. Contract amendment (exact ownership + text changes)
|
||||
|
||||
### 1.1 Homing decision — one NEW owner card + minimal responsibility-widenings
|
||||
|
||||
The layer is genuinely **new scope**. The **minimal** closed/disjoint expansion adds **one new owner card, `KBN-101-10`**, owning two brand-new, currently-unowned globs (disjoint from every existing manifest → KBN-101-06 overlap/ownerless/path-existence still pass):
|
||||
|
||||
- `packages/db/src/sink-rbac/**` — B1 layer (policy artifact + schema + evaluator + ladder + connection-selection + status-UPSERT + invariant-set + view/rule-forbid enumeration).
|
||||
- `packages/db/src/credential-handoff/**` — B2 library (SCM_RIGHTS + non-dumpable assert/verify).
|
||||
|
||||
Plus **responsibility-widenings of already-owned files** (no manifest-glob change → no overlap):
|
||||
|
||||
| Existing card | Already-owned path(s) touched | Widened responsibility (v4; v3 items preserved) |
|
||||
| ---------------------------------------- | --------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `KBN-101-00` | `infra/pg-bootstrap/roles.sql`, `infra/pg-bootstrap/tests/**` | **[GA-1 FIX]** Create the **fixed rung-ROLES** (3 LOGIN roles + their capability roles) with **deny-by-default base attributes** and **CONNECT** grants; bootstrap catalog tests that each rung role carries the safe attributes (F2) and is **non-owner** of the sink. **[AD-1 FIX]** Amend the frozen §4 **L176 CONNECT-allowlist certification** so the two new LOGIN rungs (`mosaic_runtime_user`, `mosaic_runtime_orchestrator`) are admitted to the `CONNECT` allowlist **and asserted `TEMPORARY`-denied**; otherwise the frozen L176 cert test REDS (it fails if an app login retains TEMPORARY, and it enumerates the CONNECT allowlist). **[NB-5 FIX]** Grant the two new capability roles (`mosaic_runtime_user_capability`, `mosaic_runtime_orchestrator_capability`) the **baseline frozen §4 L176 runtime grants** they need to function: `USAGE ON SCHEMA mosaic`, `USAGE ON SCHEMA drizzle` + `SELECT ON drizzle.__drizzle_migrations, drizzle.__mosaic_migration_manifest` (ledger read), and the relevant sequence `USAGE`/`SELECT` grants — mirroring what `mosaic_runtime_capability` already holds; without these the two new rungs cannot connect/operate and would FAIL the frozen §6 L222 runtime verify (missing inherited capability/sequence/ledger grants → fail-closed). **It owns NO DDL on `tasks` or the override table** — those relations do not exist at bootstrap time, so RLS/column-grant/policy/FORCE/unique-key/trigger DDL **cannot** live here. |
|
||||
| `KBN-100` (producer; SPEC target of -10) | its own `packages/db/src/schema.ts` + generated `packages/db/drizzle/**` migration + migration tests | **[GA-1 + GA-2/Q2 + B-1 + B-2 + B-3 + B2-INSERT FIX]** Homes ALL `tasks`-specific sink DDL **and the new `task_status_write_override` table DDL** **after** it creates `tasks`, per the frozen §4 L178 producer/consumer precedent: the rung column-grant matrix (F1/F7/N-3), the **RLS `WITH CHECK` write-source policies** (Q2/F1/N-3), **[B2-INSERT] the workspace-predicate `workspace_id = current_setting('mosaic.workspace_id', true)::uuid` as a conjunct of EVERY rung's INSERT enforced check on `tasks` — realized as a single compound `WITH CHECK` OR `AS RESTRICTIVE`, NEVER as separate permissive policies (v5 composition mandate, §1.4 item 3-bis) — (and thus into the UPSERT INSERT arm), and the symmetric INSERT workspace binding on `task_status_write_override`** so tenant isolation is homed at the sink on the write path too, the **per-rung `SELECT`/`UPDATE` `USING` tenant policies** (B-2), the **override-table subquery** in every `tasks` write policy (B-1), the `tasks (workspace_id,id)` **UNIQUE** candidate key (F4), the `actor_id` attribution metadata column (Q1 audit channel), the **no-status-normalizing-trigger / no-status-view / no-status-rule** invariants (F3/N-1/N-2) — **[NB-1] extended to cover the `task_status_write_override` relation as well** (no status-writing trigger/RULE/SECURITY DEFINER function on the override relation may write `tasks.status`), and — as the **TERMINAL migration step** — `ENABLE`/`FORCE ROW LEVEL SECURITY` on `tasks` and on the override table, run strictly **AFTER** all expand/backfill/repair DML (B-3). **KBN-100's DDL scope statement is explicitly amended** to include RLS + the override table on `tasks` under Mos Q2 coordinator co-authorization (§1.4). Certified deployed by KBN-101-09. |
|
||||
| `KBN-101-01` | `packages/db/src/connection-identity.ts`(+spec), `apps/gateway/src/database/database.module.ts`(+spec) | Provision **only the three generic rung pools** at boot; set the per-request **workspace session predicate** (B-2) on each checked-out connection. **[F2 FIX]** Extend the frozen L222 unsafe-attribute checkout assertion to **each** rung connection: effective role == authorized rung **AND** fail closed on SUPERUSER / CREATEROLE / CREATEDB / REPLICATION / **BYPASSRLS** / ownership of the sink tables (`DATABASE_ROLE_UNSAFE`). **[AD-2/N-5 FIX]** -01 **provisions pools only**; it does **not** import -10. The rung→pool SELECTION lives in -10's `connection-selection.ts`, which **consumes** -01's provisioned pools (import direction `10→01`; §2.2). |
|
||||
| `KBN-101-03` | `packages/db/src/migrator/**` (glob), **`docker/db-migrator.Dockerfile`** | Consume -10's credential-handoff: **SCM_RIGHTS-only** fd acquisition + **non-dumpable re-verify** after the drop to `10003:10003`. **[GA-4(ii) FIX]** The root-only **migrator launch wrapper** homes here (the migrator image). |
|
||||
| `KBN-101-05` | `apps/gateway/Dockerfile`(+spec), renderer secret matrix | Renderer mounts **three** rung DSN secrets (`mosaic-db-runtime-url` + `mosaic-db-runtime-user-url` + `mosaic-db-runtime-orchestrator-url`), each `0600 10001:10001`, Gateway-only. Gateway rung-connection selection is a **DB-role choice, not a UID drop**, so -05 owns **no** SCM_RIGHTS/dumpable behavior (moved to -03 per GA-4(ii)). |
|
||||
| `KBN-101-02` | `packages/storage/src/{cli,migrate-tier}.ts`, `packages/storage/src/adapters/postgres.ts` (already-owned) | Importer privileged-fd case uses SCM_RIGHTS (extends existing safe-open, drop to `10002:10002`); status writes go through the -10 status-preserving-UPSERT helper. |
|
||||
| `KBN-101-06` | `tools/ci/fixtures/kbn101-ddl-inventory.json`, `.woodpecker/ci.yml` | Add `KBN-101-10` to the inventory fixture/command matrix; add the rung-selection + SCM_RIGHTS + **per-tier RLS negatives** + **override-veto negative** + **cross-workspace read negative** + **[B2-INSERT] cross-workspace INSERT/UPSERT negative** + **NULL-GUC INSERT negative** + **owner/backfill-ordering** cases to the matrix. **[Gate A-delta-2 obs #2 / N-2 continuous — ADOPTED]** Home the **continuous no-status catalog scan as a -06 CI gate that runs on EVERY migration** (promoted from the v3 one-shot -09 recommendation): the scan asserts no status-writing trigger/RULE/`SECURITY DEFINER` function/view on `tasks` **or on `task_status_write_override`** (NB-1), so a future migration that adds a status-write path fails CI, not only the one-time cert. |
|
||||
|
||||
### 1.2 §7 manifest expansion (KBN-101-06 must still pass)
|
||||
|
||||
Add exactly one row to the §7 card table (complete, disjoint ownership universe):
|
||||
|
||||
> | `KBN-101-10` declarative sink-RBAC + per-role connection-selection + credential-handoff | 00,01 | **Only:** `packages/db/src/sink-rbac/**`, `packages/db/src/credential-handoff/**`. It homes: the source-controlled declarative RBAC policy artifact + schema + evaluator; the fixed **rung ladder** + federation-aware identity→rung resolver; the rung→DB-credential **connection-selection** module (consumes -01 pools; import direction `10→01`); the **status-preserving-UPSERT** contract + **invariant field-set** module + **no-status view/rule/trigger enumeration**; and the **SCM_RIGHTS** credential-handoff + **non-dumpable** assert/verify library. It creates no roles, no DDL, no Compose/CI/deploy path; it is consumed by -00 (role names), -01 (connection wiring + identity/attribute verify), -02/-03/-05 (write path + fd handoff), specified-against by **KBN-100** (sink DDL: grants/RLS/`USING`+`WITH CHECK` policies/override-table/FORCE/unique-key/trigger-view-rule-invariant on `tasks`), and certified deployed by KBN-101-09. |
|
||||
|
||||
Amend the §7 L245 closing sentence (added clause **bold**): "The manifests above are the complete ownership universe for KBN-101 implementation paths **including KBN-101-10**; the KBN-101-06 inventory test fails on overlap, an ownerless in-scope path, or a nonexistent declared path."
|
||||
|
||||
**Dependency edges (v3, unchanged from v2 — acyclic; -10 depends only on 00,01):**
|
||||
|
||||
- `KBN-101-10 depends on 00,01`
|
||||
- `KBN-101-02 depends on 01,03,10`
|
||||
- `KBN-101-03 depends on 00,01,10`
|
||||
- `KBN-101-05 depends on 00,03,10`
|
||||
- `KBN-101-06 depends on 02,03,05,07,10`
|
||||
- `KBN-101-08 depends on 00…07,10`
|
||||
- `KBN-101-09 depends on KBN-100,08` (unchanged; evidence-only)
|
||||
|
||||
**Why still disjoint (KBN-101-06 green — UNCHANGED from v2):** the only NEW ownership is the two -10 globs. The B-1 override table, B-2 `USING` policies, and B-3 terminal-FORCE ordering are **all producer DDL homed in KBN-100's already-owned `schema.ts`/`drizzle/**`** (a *responsibility* widening on files KBN-100 already owns, exactly like the `tasks`sink DDL). The N-1/N-5 additions live inside -10's already-declared`sink-rbac/**`glob. AD-1 amends -00's already-owned`roles.sql`/tests. **No glob is split, narrowed, or shared → KBN-101-06 overlap/ownerless/path-existence stay green.\*\*
|
||||
|
||||
### 1.3 §4 identity-model amendment (per-role, closed additive set — F2 hardened; AD-1 CONNECT cert)
|
||||
|
||||
Add these rows to the §4 role table (§4 L141-151). **Per-ROLE, not per-user.** Implements Jason's B1 (User = god; main Orchestrator near-god, User-overridable; others deny-by-default):
|
||||
|
||||
| Role (added) | Attributes (F2-hardened) | Membership / session use |
|
||||
| ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `mosaic_runtime_user_capability` | `NOLOGIN`, no ownership/admin attrs. | Holds the **User (god)** rung's named grants incl. `INSERT(status)`, `UPDATE(status)` + the status-transition RLS `WITH CHECK` policy on `tasks`; **plus the SOLE `INSERT/UPDATE/DELETE` grant on `task_status_write_override`** (B-1); never granted to Orchestrator/base. |
|
||||
| `mosaic_runtime_orchestrator_capability` | `NOLOGIN`, no ownership/admin attrs. | Holds the **Orchestrator (near-god)** rung's grants; identical to User **except** (i) every `tasks` write policy **subqueries** the override table and is REJECTED at the sink when an active User deny row exists for that `(workspace_id, task_id, orchestrator)` (B-1); (ii) `INSERT(status)` is pinned by `WITH CHECK` to a **non-terminal initial status** (N-3) — only User may INSERT an arbitrary/terminal status; (iii) it has only `SELECT` (never write) on the override table. Scope is tier/task-identity only (F6). |
|
||||
| `mosaic_runtime_user` | **`LOGIN INHERIT NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`**, no ownership; **non-owner of every sink table + the override table**. | Member of `mosaic_runtime_user_capability` only (`WITH INHERIT TRUE, SET FALSE, ADMIN FALSE`); the connection-selection **User-rung** credential. **[AD-1]** Requires `CONNECT`, denied `TEMPORARY`. |
|
||||
| `mosaic_runtime_orchestrator` | **`LOGIN INHERIT NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`**, no ownership; **non-owner of every sink table + the override table**. | Member of `mosaic_runtime_orchestrator_capability` only (same INHERIT/SET/ADMIN flags); the **Orchestrator-rung** credential. **[AD-1]** Requires `CONNECT`, denied `TEMPORARY`. |
|
||||
|
||||
The existing `mosaic_runtime` is confirmed already `LOGIN INHERIT` with no admin attrs (frozen L150) and non-owner; it becomes the **others / deny-by-default** base rung. On the sink it retains `SELECT` (workspace-scoped `USING`, B-2), `INSERT (explicit non-status column list)` (F1), and column-level `UPDATE` on **non-status, non-invariant, non-key, non-tenancy** columns only (F7), each `USING` workspace-scoped; it has **no** `INSERT(status)`, **no** `UPDATE(status)`, **no** `DELETE`, **no** `TRUNCATE` on `tasks`, **no** write on the override table, and RLS denies its status/override writes by default. The rung set is **fixed at three**.
|
||||
|
||||
**F2 note:** the three rung LOGIN roles' `NOBYPASSRLS` + `NOSUPERUSER` + **non-ownership** are load-bearing — `FORCE ROW LEVEL SECURITY` collapses for any SUPERUSER/BYPASSRLS role and RLS is silently bypassed by a table **owner**. These attributes are declared in -00 bootstrap, catalog-proven in -00 tests, and re-verified at every checkout by -01 (extended L222 assertion).
|
||||
|
||||
**[AD-1] CONNECT-allowlist certification amendment (frozen §4 L176):** L176 today grants `CONNECT` only to `mosaic_runtime`, `mosaic_migrator`, and the bootstrap actor, and its cert **fails if an unrelated login retains `CONNECT` or either application login retains `TEMPORARY`.** The two new LOGIN rungs are additional application logins that require `CONNECT` and must be `TEMPORARY`-denied. The -00 row + rc.20 text **explicitly amend the L176 allowlist** to `{mosaic_runtime, mosaic_runtime_user, mosaic_runtime_orchestrator, mosaic_migrator, bootstrap-actor}`, and the -00 cert asserts `TEMPORARY` denial for all three runtime rungs. Without this amendment the frozen -00 cert test REDS on the two new logins.
|
||||
|
||||
**rc.20 amendment text (v3 — extends v2)** — amend §4's closing note and SHARED-CONTRACT rc.5 L82/L88-90 ("Non-effect: role graph … unchanged / neither creates roles/secrets") with:
|
||||
|
||||
> **rc.20 (Envelope A) — authorized by Jason's declarative-RBAC B1 ruling + Mos OPTION A + Mos Q1 (TIER-LEVEL) + Mos Q2 (RLS):** adds (i) the fixed **User/Orchestrator/others** runtime rung-roles (per-ROLE, deny-by-default); each rung **LOGIN** role is `NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`, **NON-OWNER** of every sink/override table, **added to the §4 L176 `CONNECT` allowlist**, and **asserted `TEMPORARY`-denied**. **No per-federated-user role or credential is created** — federated identity maps to a rung **at authorization time only**, and per-writer **attribution lives in metadata columns** (app-supplied `actor_id` on `task_events`), **never** in DB credentials/roles. (ii) Two new fixed rung DSN secrets **`mosaic-db-runtime-user-url`** and **`mosaic-db-runtime-orchestrator-url`** (Gateway-only, `0600 10001:10001`), extending the frozen runtime-URL secret row. (iii) **RLS `WITH CHECK` write-source policies (including the B2-INSERT per-rung INSERT tenant binding `workspace_id = current_setting('mosaic.workspace_id', true)::uuid`, composed as a SINGLE compound `WITH CHECK` or `AS RESTRICTIVE` — never multi-permissive-intended-to-AND, per the v5 composition mandate §1.4 item 3-bis) + per-rung `SELECT`/`UPDATE` `USING` tenant policies + `FORCE ROW LEVEL SECURITY`** on the sink table `tasks` **and on the new sink-resident `task_status_write_override` table** (tighten-only, deny-by-default) — a §10 schema-v1 **mechanism addition** beyond the prior grant/revoke-only model, authorized by Mos Q2 as coordinator co-authorization with Jason's B1. **The two new rung capability roles additionally receive the baseline frozen §4 L176 runtime grants (`USAGE ON mosaic`, `drizzle` ledger `SELECT`, sequence grants) [NB-5] so they satisfy the frozen §6 L222 runtime verify.** (iv) The **`task_status_write_override` table** (keyed `(workspace_id, task_id, tier)`, no per-writer key), written only by the User rung, read by the Orchestrator/base rung `tasks` write-policy subquery, homes the User-override **at the sink**. The sink DDL (grants + RLS `USING`/`WITH CHECK` + override table + FORCE + unique key + trigger/view/rule invariants) is homed in the **KBN-100 producer** after it creates `tasks`, with `FORCE RLS` as the **terminal** migration step after all backfill/repair DML, per the §4 L178 producer/consumer precedent.
|
||||
|
||||
> **Frozen-invariant note (authorization basis):** touching the frozen role graph, the §4 L176 CONNECT allowlist, and adding RLS + the override table are §10 contract-change classes (MISSION-MANIFEST §10; SHARED-CONTRACT L82/L88-90; frozen §4 L176). They are authorized here by **Jason B1 + Mos OPTION A + Mos Q1 + Mos Q2**. The addition is strictly a **fixed per-ROLE** set + a **tighten-only** RLS mechanism + a **task/tier-scoped** override table. It stops exactly at the HALT boundary: **no** per-federated-user topology is introduced (§5 proof).
|
||||
|
||||
### 1.4 §10 mechanism amendment — RLS `WITH CHECK` (write-source + INSERT tenant-bound) + `USING` + sink-resident override at the sink (Q2), with the two mandatory conditions
|
||||
|
||||
The frozen write-authority model is grant/revoke-only; **RLS appears nowhere in the frozen contract.** Mos Q2 **AUTHORIZES** adding declarative, deny-by-default, **enforce-AT-THE-SINK, fail-closed** RLS policies, because grant/revoke alone cannot express (a) row-level _write-source_ enforcement (pin new-row status, F1/N-3), (b) a _sink-enforced_ User-override that the Orchestrator-rung path cannot skip (B-1), (c) _tenant-scoped reads/updates_ (B-2), or (d) **[B2-INSERT] _tenant-scoped INSERTs_ — binding `workspace_id` on the write/create path**, which neither column grants nor the `USING` read policy can do (PostgreSQL does not apply `USING` to INSERT). RLS is **tighten-only**: it can only further restrict beyond the column grants, never widen them.
|
||||
|
||||
**KBN-100 producer DDL scope is explicitly amended** to include, on `tasks` (and, where noted, the override table):
|
||||
|
||||
1. `ALTER TABLE tasks ENABLE ROW LEVEL SECURITY;` — and the same on `task_status_write_override`.
|
||||
2. **CONDITION (b) — EXPLICIT DECISION: `ALTER TABLE tasks FORCE ROW LEVEL SECURITY;`** (and on the override table) — DECIDED **YES/ON**, stated explicitly and not left defaulted, so that **even the table owner (`mosaic_schema_owner`) and any maintenance connection cannot silently bypass RLS**. (Directly closes the live U-Connect failure mode where an owner connection with `rls_forced=false` silently VOIDED RLS.)
|
||||
3. A **deny-all default** posture (no permissive policy) plus per-rung policies:
|
||||
- **Write (`WITH CHECK`):** base/others rung → new-row `status = <initial state 'backlog'>` only, no status UPDATE; **Orchestrator rung → status INSERT pinned to a non-terminal initial status (N-3), and every write (`INSERT`/`UPDATE`/transition) additionally `WITH CHECK`'d against `NOT EXISTS(<active User deny row for (workspace_id, id, 'orchestrator')>)` (B-1)**; User rung → status INSERT (any legal status) / transition. Every write policy is `WITH CHECK` (write-source), tighten-only.
|
||||
- **[B2-INSERT — tenant isolation on the write path] Every rung's INSERT `WITH CHECK` on `tasks` MUST include the workspace predicate `workspace_id = current_setting('mosaic.workspace_id', true)::uuid` as a conjunct of the _same_ enforced check that carries the status-pin (and, for the Orchestrator rung, the B-1 override subquery) — see the COMPOSITION MANDATE (item 3-bis) for the exact realization.** This is REQUIRED because PostgreSQL does **NOT** apply `USING` to INSERT, and the explicit status-pinning `WITH CHECK` **suppresses** the `USING`→`WITH CHECK` substitution — so in v3 `workspace_id` was **unbound on the INSERT path** and any rung could `INSERT INTO tasks (workspace_id, …) VALUES (<foreign W2>, …)` (directly or via the sanctioned status-upsert INSERT arm) into a victim workspace, defeating B-2's "tenant isolation homed at the sink" goal and evading SHARED-CONTRACT §7.1 (body workspace forbidden). **CORRECTED SEMANTICS (v5 — the delta-3 fix):** multiple **PERMISSIVE** policies for the same command combine with **OR** — a row is admitted if it satisfies **ANY** one permissive policy's `WITH CHECK`, ANDed with all `AS RESTRICTIVE` policies' checks. "All applicable `WITH CHECK` clauses must pass / they AND" is **FALSE as a general rule** and is struck from this envelope; the AND-conjunction the B2-INSERT closure depends on holds ONLY (a) _within a single policy_ whose `WITH CHECK` is one AND-conjoined expression, or (b) _across `AS RESTRICTIVE` policies_. Realizing the status-pin, the workspace-bind, and the override as separate _permissive_ policies would let a foreign-workspace INSERT that satisfies only the status-pin be admitted by OR — re-opening B2-INSERT/B-1 — which item 3-bis forbids. **Fail-closed on NULL:** an absent GUC makes `current_setting(...,true)` return NULL, the equality yields NULL (not true), and — provided the predicate is a conjunct of the single enforced check (item 3-bis) — the INSERT **ERRORS at the sink**. The **same INSERT workspace binding is applied symmetrically to `task_status_write_override`**, under the identical composition mandate (its own single compound `WITH CHECK` or `AS RESTRICTIVE`).
|
||||
- **Read/lock (`USING`) — B-2:** per-rung `SELECT` and `UPDATE` `USING` policies **workspace-scoped** by the session predicate `workspace_id = current_setting('mosaic.workspace_id', true)::uuid` (set per request by -01, §2.2). **`USING(true)` is explicitly FORBIDDEN** — tenant isolation rides on this predicate; a lazy `USING(true)` would expose every workspace's rows to every rung and is a fail-closed contract violation caught by a negative test.
|
||||
- **3-bis. [v5 — LOAD-BEARING] COMPOSITION MANDATE (how the sink check MUST be realized; both delta-3 gates prescribed it).** Per rung, per command (INSERT / UPDATE / transition), the sink enforcement of the status-pin **AND** the workspace-bind **AND** (Orchestrator rung only) the User-override subquery MUST be realized as EITHER:
|
||||
- **(a) a SINGLE policy** whose `WITH CHECK` is the _full conjunction_ — `<status-pin> AND workspace_id = current_setting('mosaic.workspace_id', true)::uuid` and, for the Orchestrator rung, `AND NOT EXISTS(<active User-override deny row for (workspace_id, id, 'orchestrator')>)`; **OR**
|
||||
- **(b)** the status-pin as the **SOLE PERMISSIVE** policy for that command, **plus** the workspace-bind and (Orchestrator) the override subquery authored **`AS RESTRICTIVE`** (restrictive policies AND with everything, so a row must pass every restrictive check _and_ at least one permissive check).
|
||||
- **EXPLICITLY FORBIDDEN:** realizing the status-pin, the workspace-bind, and the override as **multiple _permissive_ policies whose predicates are intended to AND.** Permissive policies combine with **OR**, so that realization admits a foreign-workspace INSERT that satisfies only the status-pin (cross-tenant injection) and equally bypasses the B-1 override — exactly the delta-3 defect. This mandate applies to the `tasks` INSERT/UPDATE/transition policies **AND** to the `task_status_write_override` table's own policies (same single-compound-`WITH CHECK`-or-`AS RESTRICTIVE` requirement; never multi-permissive-intended-to-AND). The producer DDL and the -09/-06 certification assert the realized policies are either single-compound or `AS RESTRICTIVE` (never separate permissive policies expected to AND) — a lint/catalog check over `pg_policy.polpermissive` backs it.
|
||||
- **[v6 — N1: strengthened `pg_policy.polpermissive` lint (supplementary, NOT sufficient).]** The `pg_policy.polpermissive` lint (continuous -06 scan, §2.7) MUST (a) count permissive policies **grouped by EFFECTIVE role — including `PUBLIC` and role inheritance**, not merely by the policy's named `TO` role: a `TO PUBLIC` (or inherited-through-a-capability-role) permissive policy applicable to a rung's command is counted **alongside** that rung's status-pin, because at evaluation time it OR-widens exactly like a second same-role permissive policy — so a `TO PUBLIC` permissive next to a rung's status-pin is a **lint failure**; and (b) ideally **reject an internally-disjunctive single `WITH CHECK`** — a single policy whose expression is itself `status_pin OR workspace_bind` satisfies the naive one-permissive-policy count yet violates the composition mandate exactly as two permissive policies would, so the lint should detect a top-level `OR` between the status-pin and the workspace-bind/override conjuncts within one policy expression. **The lint is SUPPLEMENTARY, not sufficient:** the **behavioral positive-controls remain the PRIMARY proof** of correct composition (dropping the workspace conjunct — resp. the override subquery — alone flips the hardened negative to a PASS; §2.4 clause 5 / §2.9). The lint backstops the catalog shape; it does not replace the behavioral proof.
|
||||
- **[v6 — N2: option-(b) vacuous-deny guard invariant — ≥1 permissive per writable (rung, command).]** Under option (b) (status-pin as the **sole PERMISSIVE** policy + workspace-bind/override `AS RESTRICTIVE`), PostgreSQL admits a row only if **≥1 PERMISSIVE `WITH CHECK` is TRUE AND all RESTRICTIVE checks pass** — so if a writable (rung, command) has **zero** permissive policies, every restrictive-only evaluation denies and the command **over-denies / fails closed** (nothing can ever be written). Therefore there **MUST be ≥1 PERMISSIVE policy per writable (rung, command)** — this "≥1 permissive per writable (rung, command)" is the **load-bearing option-(b) guard invariant** (the workspace-bind/override being restrictive is safe _only_ because the status-pin supplies the required permissive). It is verified by the mandated **"same-workspace valid-status write succeeds" positive tests** for each writable command — INSERT, transition/UPDATE, and SELECT — which would FAIL (spurious over-deny) if the permissive were dropped, and by the N1 lint confirming exactly one permissive status-pin (grouped by effective role) is present per writable (rung, command).
|
||||
4. **B-3 migration ordering (TERMINAL FORCE):** all expand/backfill/repair DML on `tasks` (SHARED-CONTRACT §5.1 phase-3 L165-171, §5.3 L188) runs **FIRST**, while `tasks` has RLS **disabled**; the `ENABLE`/`FORCE ROW LEVEL SECURITY` statements are the **TERMINAL** DDL of the activation migration. **No standing `mosaic_schema_owner` ALLOW write policy exists after activation** (that would recreate the unconstrained owner write path = U-Connect mode). Any schema_owner maintenance policy, if ever needed, is **migration-only** and must be proven **unreachable by the 3 runtime LOGIN roles** + a negative test.
|
||||
|
||||
**CONDITION (a):** every runtime per-role LOGIN role (`mosaic_runtime`, `mosaic_runtime_user`, `mosaic_runtime_orchestrator`) **MUST remain NON-OWNER** of every sink table **and the override table** (owner stays `mosaic_schema_owner`). Enforced by -00 bootstrap catalog assertion + re-verified at each checkout by -01 (F2). A rung role that owns a sink/override table is a fail-closed contract violation.
|
||||
|
||||
**Break-glass boundary (B-3, named explicitly in KBN-101-09 cert):** the ONLY identities that legitimately bypass `FORCE RLS` are the sanctioned break-glass actors named in the frozen contract — `mosaic_extension_owner` (`NOLOGIN SUPERUSER`, §4.1 L146) and the external platform bootstrap actor (superuser, §4/§7; §9 L266 residual authority). Both are `NOLOGIN`/external, carry no runtime credential, and are outside application containment. The -09 cert **names these two as the accepted, audited boundary** — not a new hole — and asserts the three runtime LOGIN rungs are NOT superuser/NOT BYPASSRLS and cannot assume either.
|
||||
|
||||
**[NB-4 — owner-toggle-FORCE containment is NON-REACHABILITY, not incapability.]** The -09 break-glass enumeration must **additionally name `mosaic_schema_owner`** (frozen §4 L145, `NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS`) as a **contained-but-CAPABLE** path. A table **owner** — regardless of `NOSUPERUSER`/`NOBYPASSRLS` — can `ALTER TABLE tasks DISABLE ROW LEVEL SECURITY` or `ALTER TABLE tasks NO FORCE ROW LEVEL SECURITY` and then write freely; `FORCE`/`NOBYPASSRLS` do **not** make an owner incapable. So the containment claim is **NOT** "the owner cannot bypass RLS" — it is **"the owner role is NOT REACHABLE from the three runtime LOGIN rungs."** The -09 cert asserts this non-reachability **explicitly**: (i) `mosaic_schema_owner` is `NOLOGIN` (no direct credential); (ii) it is reachable only via `mosaic_migrator`'s `SET ROLE` (frozen §4 L147, migrator runner-before-Gateway, `max:1` locked migration session), which is **not** one of the three runtime rungs and holds no runtime pool; (iii) none of `mosaic_runtime` / `mosaic_runtime_user` / `mosaic_runtime_orchestrator` is a member of, or can `SET ROLE` to, `mosaic_schema_owner` or `mosaic_migrator` (catalog-proven at -00, re-verified at every checkout by -01's L222 non-owner/assumability assertion). **[NB-4-transitive — Gate B-delta-3 nit, folded]** the non-reachability assertion in the -09 cert MUST be stated as **TRANSITIVE membership**, not merely the absence of a **direct** `pg_auth_members` row: PostgreSQL's ownership/privilege check returns true for an **inherited** membership reached _through a capability role_, which would confer the owner's `ALTER TABLE … NO FORCE` / `DISABLE ROW LEVEL SECURITY` right. The assertion therefore uses the recursive predicates — `NOT pg_has_role('mosaic_runtime_user', 'mosaic_schema_owner', 'USAGE')` (and the same for each runtime rung against `mosaic_schema_owner` and `mosaic_migrator`), equivalently `NOT has_privs_of_role(<rung>, 'mosaic_schema_owner')` — which fold in transitive/inherited membership through any intermediate (capability) role. The frozen §6 L222 fail-closed-on "owner/migrator identity **or assumability**" check plausibly already covers this transitive case; v5 states "**transitive membership**" **explicitly** in the -09 cert assertion so no implementation reads "non-member" as only the direct-edge check. **[v6 — F-NB4: cover SET-ROLE assumability, not only inherited privilege.]** The v5 predicates `NOT pg_has_role(<rung>, 'mosaic_schema_owner', 'USAGE')` / `NOT has_privs_of_role(<rung>, 'mosaic_schema_owner')` capture the **inherited-privilege** axis, but a membership granted `WITH INHERIT FALSE, SET TRUE` yields `has_privs_of_role = FALSE` **yet still allows `SET ROLE mosaic_schema_owner`** followed by `ALTER TABLE … NO FORCE ROW LEVEL SECURITY` — an assumability path the `USAGE`/`has_privs_of_role` predicate alone does **not** close. The -09 cert assertion therefore **conjoins the `MEMBER` variant** — `NOT pg_has_role(<rung>, 'mosaic_schema_owner', 'MEMBER')` **AND** `NOT pg_has_role(<rung>, 'mosaic_migrator', 'MEMBER')` — for **each of the three runtime rungs** (`mosaic_runtime`, `mosaic_runtime_user`, `mosaic_runtime_orchestrator`); the `MEMBER` privilege captures SET-ROLE reachability **regardless of `INHERIT`**, so `pg_has_role(rung, owner, 'USAGE')` **AND** `pg_has_role(rung, owner, 'MEMBER')` are asserted false together. This is stated explicitly rather than leaning on "frozen §6 L222 assumability plausibly covers it." (The intended role graph is already safe — per envelope §1.3 + frozen §4 L150, each rung is a member of **only** its own capability role, `WITH INHERIT TRUE, SET FALSE, ADMIN FALSE`, and of no owner/migrator role — so no live `SET TRUE` edge to the owner/migrator exists; this fold is **assertion-completeness hardening, not a live hole.**) The owner's toggle-then-write capability is therefore real but confined to the migrator/owner phase (frozen §5, runner-before-Gateway), never to a live runtime rung — that is the containment, stated as **transitive non-reachability** rather than incapability.
|
||||
|
||||
**Red-team requirement (baked into §3/§4; v5-HARDENED so the negatives cannot false-green under the OR-trap):** each per-rung policy is proven **deny-by-default via NEGATIVE tests PER TIER** in real PostgreSQL:
|
||||
|
||||
- an unauthorized write-source (base terminal-status INSERT/transition) that **ERRORS**;
|
||||
- **[B2-INSERT — hardened] a cross-workspace INSERT/UPSERT using a row OTHERWISE VALID for the rung** — a status the rung's pin **ACCEPTS** (e.g. **base rung + `status='backlog'`**, and no active override), differing from a passing insert ONLY in `workspace_id = <foreign W2>` — so the rejection **isolates the workspace bind** (not a status/override rejection). Tested both directly and via the status-upsert INSERT arm; it **ERRORS at the sink**. **Positive-control:** removing the workspace-bind conjunct **alone** (leaving the status-pin) flips this negative to a **PASS**, proving the _workspace bind_, not the status-pin, is doing the rejecting (and proving the composition is a single enforced conjunction / RESTRICTIVE, not an OR of separate permissive policies).
|
||||
- **[B-1 — hardened] an Orchestrator write vetoed by an active User-override deny row, using a transition the Orchestrator pin OTHERWISE ACCEPTS** (a legal non-terminal transition on `(W, T)` that would succeed absent the deny row) — so the rejection **isolates the override subquery**. It **ERRORS at the sink**. **Positive-control:** removing the override subquery **alone** flips this negative to a **PASS**, proving the _override_, not the status-pin/workspace-bind, is doing the rejecting.
|
||||
- a **cross-workspace `SELECT` under the wrong/absent session predicate** → zero rows;
|
||||
- **[B2-INSERT] a NULL-GUC INSERT** (absent `mosaic.workspace_id`) that **ERRORS** (fail-closed on NULL);
|
||||
- a **same-workspace INSERT with a valid status** that **succeeds** (proving the predicate composes with F1/N-3 and does not over-deny);
|
||||
- an **owner/maintenance connection attempting a post-activation `tasks` write** that must **fail under `FORCE`**.
|
||||
All rejections **ERROR at the sink**. The two positive-controls are load-bearing: they are the direct proof that the composition is single-compound-`WITH CHECK`-or-`AS RESTRICTIVE` (delta-3), because under the forbidden multi-permissive-OR realization the hardened cross-workspace / override negatives would already PASS (false-green) without removing anything.
|
||||
|
||||
---
|
||||
|
||||
## 2. Design
|
||||
|
||||
### 2.1 Declarative RBAC policy (source-controlled artifact + schema)
|
||||
|
||||
- **Artifact:** `packages/db/src/sink-rbac/policy.task-status.v1.ts` — a source-controlled, versioned, deny-by-default policy object; the single source of truth for "who may write the sink." No policy lives in the database except the _derived_ GRANT/RLS state the KBN-100 producer emits from it.
|
||||
- **Schema:** `packages/db/src/sink-rbac/policy.schema.ts` — a typed schema (zod/TS) validating: `rungs` (exactly the fixed ladder), per-rung `allow`/`deny` verbs (`insert`, `insert:status`, `update:<column-set>`, `transition:status`, `delete` — always deny), `federationMap` (federated-identity-claim → **rung**, never a role/credential per user), and `userOverride`.
|
||||
- **[F6 FIX] `userOverride` scope guardrail:** the schema **constrains** override predicates to **tier- + task-identity (row) scope `(workspace_id, task_id, tier)`** and **explicitly FORBIDS** a per-federated-writer identity as an override or authority key (the Q1 re-open trigger). A policy that names a per-federated-writer identity as an override/authority key **fails schema validation** (fail-closed at load). The override is realized as the sink-resident `task_status_write_override` table (§2.9), NOT as app-resolved state. Per-writer attribution is not an authority input; it flows only to the audit metadata column (§2.6).
|
||||
- **Evaluator:** `packages/db/src/sink-rbac/evaluate.ts` — pure `(identityClaims, requestedWrite) → Decision ∈ {allow(rung), deny(reason)}`, **deny-by-default**, tier resolution only. It **only chooses a rung**; it never mints credentials and — critically (B-1) — **it is NOT the enforcement point for the User-override**: the override is enforced in PostgreSQL by the write-policy subquery, so a compromised Gateway that skips the evaluator still cannot bypass a User deny.
|
||||
- **Ladder / federation-awareness:** `packages/db/src/sink-rbac/ladder.ts` — fixed order `User > Orchestrator > others` and `resolveRung(federatedIdentity) → rung` at authorization time (Q1: federation-awareness lives in the resolver; the sink authorizes by the writer's declared ROLE/tier).
|
||||
- **Enforcement is AT THE SINK, fail-closed:** the evaluator's rung choice only selects _which pre-provisioned connection_ is used; PostgreSQL then enforces via (a) revoked base privileges, (b) column-level `INSERT(status)`/`UPDATE(status)` granted only to the User/Orchestrator capabilities, (c) `FORCE ROW LEVEL SECURITY` + deny-all default + per-rung `WITH CHECK` policies **including the override subquery** (B-1), (d) per-rung `SELECT`/`UPDATE` `USING` **workspace predicate** (B-2, tenant isolation at the sink), (e) `DELETE/TRUNCATE` revoked from all runtime rungs, and (f) `UPDATE` on key/tenancy/invariant columns revoked from **all** rungs (F7). A mis-authorized OR override-vetoed OR cross-workspace write **fails closed in PostgreSQL** because the connection's effective DB role lacks the privilege or the policy predicate rejects the row — caller discipline is not trusted.
|
||||
|
||||
### 2.2 Per-role credential + connection-selection model (NO per-user roles; import direction pinned)
|
||||
|
||||
```
|
||||
federated writer ──(verified identity claim)──► evaluate.ts ──► rung (one of 3, fixed)
|
||||
│
|
||||
connection-selection.ts ───┤ selects the PRE-PROVISIONED
|
||||
(in -10; consumes -01) │ pool for that rung; sets
|
||||
│ SET LOCAL mosaic.workspace_id
|
||||
▼
|
||||
mosaic_runtime_user | mosaic_runtime_orchestrator | mosaic_runtime
|
||||
└── PostgreSQL enforces grants + FORCE RLS WITH CHECK(+override) + USING(workspace) ──┘
|
||||
```
|
||||
|
||||
- **`packages/db/src/sink-rbac/connection-selection.ts`** maps `rung → pooled connection`. **Exactly three** connection pools, one per rung, each opened with that rung's **fixed** mounted-secret DSN. Credentials = rungs = **3**, constant.
|
||||
- **[AD-2/N-5 — import direction pinned]** `-01` (`database.module.ts`) **provisions ONLY the three generic rung pools** at boot and knows nothing of rung _selection_. The rung→pool SELECTION lives entirely in **-10's `connection-selection.ts`**, which is **consumed by the write-path cards (`-02`/`-03`/`-05`, which already `depends on 10`)** and is **NEVER imported by `-01`**. Direction is therefore `10 → 01` (10's selection consumes 01's pools) — **acyclic; there is no `01 → 10` cycle.** An impl lane must not read "01 wires 10" as `01 → 10`.
|
||||
- **[B-2 — workspace session predicate]** on checkout, before the first query, -01 sets `SET LOCAL mosaic.workspace_id = <verified request workspace>` inside the request transaction; the per-rung `USING` policies **and the B2-INSERT `WITH CHECK`** read `current_setting('mosaic.workspace_id', true)::uuid`. The predicate is set from the **server-verified** tenant context, never from a body field (SHARED-CONTRACT §7.1 "Body workspace fields are forbidden"). A missing/invalid setting → `USING` yields no rows (fail-closed read) and INSERT `WITH CHECK` yields NULL→ERROR (fail-closed write), never cross-workspace exposure.
|
||||
- **[NB-3 — compromise scope]** the GUC is **app-set with no per-tenant DB authentication**, so B-2 (reads/UPDATE `USING`) and B2-INSERT protect a **buggy** Gateway (omitted/wrong predicate → fail-closed), NOT a **compromised** Gateway that forges `mosaic.workspace_id` to a victim tenant (accepted Mos Q1 co-resident-pool residual; defending it requires per-user DB creds = the HALT boundary). The genuine closure B2-INSERT delivers is the **buggy / unbound-INSERT** hole: even with a correct GUC, v3 left `workspace_id` unbound on INSERT so any rung could stamp a foreign workspace — now impossible.
|
||||
- **[Gate B 2(a) — poisoned-pool / SET LOCAL negative]** the `mosaic.workspace_id` GUC is folded into the frozen L174 **poisoned-pooled-session-reset + transaction `SET LOCAL` restoration** negative test (which already covers `search_path`): a negative proves `mosaic.workspace_id` is **transaction-scoped via `SET LOCAL`** (mirroring the frozen `SET LOCAL search_path` discipline, §4 L152) so a pooled connection **cannot leak a stale workspace** across requests — an un-reset/poisoned session fails closed, and a committed/rolled-back transaction does not carry the prior request's workspace.
|
||||
- **[F2 FIX] Identity + attribute verification at checkout** (`connection-identity.ts`): verify effective role == the rung's expected DB role **AND** assert the safe attributes — fail closed (`DATABASE_ROLE_UNSAFE`) on SUPERUSER, CREATEROLE, CREATEDB, REPLICATION, **BYPASSRLS**, or ownership of any sink/override table. Extends the frozen L222 check to **each** rung connection.
|
||||
- **No per-federated-user role/credential:** a new federated user needs **zero** new DB roles/credentials/secrets/pools — it resolves to an existing rung. Federation growth is O(1) in DB-role count.
|
||||
|
||||
### 2.3 Status-preserving UPSERT on a stable natural key (F4 bound)
|
||||
|
||||
- **`packages/db/src/sink-rbac/status-upsert.ts`** — the single sanctioned refresh helper: `INSERT INTO tasks (natural_key…, <mutable non-status cols…>) VALUES (…) ON CONFLICT (<natural key>) DO UPDATE SET <mutable non-status cols…>` — **`status` is never in the `DO UPDATE SET` list**, so a refresh preserves existing `status` byte-for-byte.
|
||||
- **[B-2 dependency]** the `ON CONFLICT` conflict-probe/refresh reads the existing row; under `FORCE RLS` this requires the per-rung `SELECT`/`UPDATE` `USING` policy to admit the row (workspace-scoped). Without B-2's `USING` policies the probe would see zero rows and the UPSERT would spuriously INSERT-conflict or mis-refresh — B-2 is what makes the sanctioned UPSERT actually work post-activation.
|
||||
- **[B2-INSERT dependency]** the UPSERT's **INSERT arm** is a `tasks` INSERT and is therefore subject to the B2-INSERT workspace `WITH CHECK` (§1.4/§2.4): a status-upsert that supplies a foreign `workspace_id` (or runs under an absent GUC) **ERRORS at the sink**, closing the sanctioned-UPSERT INSERT path that v3 left tenant-unbound. The helper sets `workspace_id` from the server-verified tenant context (never a body field, §7.1); it matches the session GUC by construction.
|
||||
- **[F4 FIX] Natural key:** the tenant-scoped stable identity **`(workspace_id, id)`**. No frozen `tasks(workspace_id,id)` unique key exists (only `missions_workspace_id_uidx`, rc.4 L94) — `ON CONFLICT` against a nonexistent unique index is a **HARD planning error** that fails loud. Therefore **KBN-100 MUST create `tasks (workspace_id,id)` UNIQUE** (aligned to the SI-001 pattern, SHARED-CONTRACT §5.2). **Fallback:** if KBN-100 keys `tasks` on global `id` only, the UPSERT natural key **re-binds to `(id)`** (spec-level, no HALT).
|
||||
- **Status changes only via an authorized transition:** `transitionStatus()` guarded by (a) the rung's `transition:status` policy verb, (b) `UPDATE(status)` present only on User/Orchestrator capabilities, (c) the RLS `WITH CHECK` transition policy (admits the update only as an explicit transition, never as a refresh/grant side effect) **plus the override subquery for the Orchestrator rung (B-1)**, and (d) the `USING` workspace predicate to see/lock the row (B-2). `DELETE` is denied to all runtime rungs → **DELETE + default-INSERT status reset is impossible at the sink** (fails closed on the DELETE).
|
||||
|
||||
### 2.4 INSERT-time status constraint (F1 BLOCKING + N-3 hardening)
|
||||
|
||||
Base `mosaic_runtime` must not stamp an arbitrary terminal status at creation, **and the Orchestrator rung must not create a task directly at a terminal status** (N-3). Layered fail-closed controls (homed in the KBN-100 producer DDL):
|
||||
|
||||
1. **Column-privilege:** `GRANT INSERT (<explicit non-status column list>) ON tasks TO mosaic_runtime_capability;` — the base rung **cannot name `status` on INSERT**, so `status` takes its column **DEFAULT** (`backlog`). `INSERT(status)` is granted **only** to the User/Orchestrator capabilities.
|
||||
2. **RLS `INSERT … WITH CHECK` (base):** pins **new-row `status = 'backlog'`** for the base rung.
|
||||
3. **[N-3 FIX] RLS `INSERT … WITH CHECK` (Orchestrator):** pins the Orchestrator rung's new-row status to a **non-terminal initial status** (`status IN {backlog, ready}`; never `done`/`cancelled`/`in_review`). **Only the User (god) rung may INSERT an arbitrary/terminal status.** This prevents an Orchestrator creating a task directly at `done`/`cancelled`, skipping the transition/lease/review trail (SHARED-CONTRACT §7.2 `POST /tasks` L272 vs `POST /tasks/:taskId/transition` L275). The exact terminal/non-terminal set is bound by KBN-100 to the §3 L134 vocabulary.
|
||||
4. **[B2-INSERT FIX] RLS `INSERT … WITH CHECK` (EVERY rung, tenant binding):** every rung's INSERT sink-check MUST include `workspace_id = current_setting('mosaic.workspace_id', true)::uuid` as a conjunct of the **same enforced check** that carries the status-pin (base=`backlog`, Orch=non-terminal, User=any) **and** the B-1 override subquery on the Orchestrator rung. `status` is bound by clause 2/3; `workspace_id` is bound by this conjunct; they are orthogonal columns so there is no conflict. **[v5 — corrected PostgreSQL semantics; the delta-3 fix]** the earlier justification "because PostgreSQL requires all applicable `WITH CHECK` clauses to pass, they AND" is **FALSE and is struck**: multiple **PERMISSIVE** policies for a command combine with **OR** (a row is admitted if it satisfies **ANY** one permissive `WITH CHECK`, ANDed with all `AS RESTRICTIVE` checks). The AND this closure needs holds ONLY (a) within a _single_ policy whose `WITH CHECK` is one AND-conjoined expression, or (b) across `AS RESTRICTIVE` policies. **Fail-closed on NULL:** an absent GUC → NULL equality → INSERT ERRORS — _provided_ the workspace conjunct sits in the single enforced check (clause 5). This closes the v3 gap where the explicit status `WITH CHECK` suppressed the `USING`→`WITH CHECK` substitution, leaving `workspace_id` unbound on INSERT and allowing any rung to INSERT into a foreign workspace.
|
||||
5. **[v5 COMPOSITION MANDATE — LOAD-BEARING] Single-compound-`WITH CHECK`-or-`AS RESTRICTIVE`, never multi-permissive-intended-to-AND.** The status-pin (clause 2/3), the workspace-bind (clause 4), and (Orchestrator) the B-1 override subquery MUST be realized per rung, per command as EITHER **(a)** a **SINGLE policy** whose `WITH CHECK` is the full conjunction `<status-pin> AND workspace_id = current_setting('mosaic.workspace_id', true)::uuid [AND NOT EXISTS(<active override deny row>)]`; **OR (b)** the status-pin as the **sole PERMISSIVE** policy **plus** the workspace-bind and (Orchestrator) the override authored **`AS RESTRICTIVE`**. Realizing them as **multiple _permissive_ policies intended to AND is EXPLICITLY FORBIDDEN** — permissive policies OR, so a foreign-workspace INSERT satisfying only the status-pin (e.g. base + `backlog` + foreign `workspace_id`) would be OR-admitted, re-opening B2-INSERT, and an override-vetoed Orchestrator write satisfying the status-pin would likewise slip the B-1 veto. The **identical mandate applies to `task_status_write_override`'s own policies** (single compound `WITH CHECK` or `AS RESTRICTIVE`; never multi-permissive-AND). A `pg_policy.polpermissive` catalog check + the hardened positive-control negatives (below) prove the realization. **[v6 — N1]** the `pg_policy.polpermissive` check is strengthened to count permissive policies **grouped by EFFECTIVE role (including `PUBLIC` and role inheritance)** — a `TO PUBLIC`/inherited permissive applicable to a rung's command counts alongside its status-pin — and to **reject an internally-disjunctive single `WITH CHECK`** (a lone policy whose expression is `status_pin OR workspace_bind` OR-widens exactly like two permissive policies); the lint is **supplementary, and the behavioral positive-controls below remain the PRIMARY proof** (§1.4 item 3-bis). **[v6 — N2]** because option (b) admits a row only when **≥1 permissive `WITH CHECK` is TRUE AND all restrictive checks pass**, there **MUST be ≥1 PERMISSIVE policy per writable (rung, command)** — the load-bearing option-(b) vacuous-deny guard — else the command over-denies (fails closed); this invariant is proven by the mandated "same-workspace valid-status write succeeds" positive tests (INSERT, transition/UPDATE, SELECT).
|
||||
|
||||
**Red-first tests (v5-hardened):** base-rung `INSERT … status='done'` **ERRORS** (column-privilege and/or RLS); base-rung INSERT omitting status succeeds at DEFAULT; **Orchestrator `INSERT … status='done'` ERRORS (N-3)**, Orchestrator `INSERT … status='backlog'` succeeds; User-rung `INSERT … status='done'` succeeds. **[B2-INSERT — hardened valid-status form]** the cross-workspace negative uses a row **otherwise valid for the rung** — **base rung + `status='backlog'` + `workspace_id=<foreign W2>`, no active override** — differing from a passing insert ONLY in `workspace_id`, so the **ERROR** isolates the _workspace bind_ (not a status/override rejection); tested direct and via the status-upsert INSERT arm. **Positive-control:** removing the workspace-bind conjunct **alone** flips this to a **PASS** (proving the bind — not the status-pin — rejects, hence a single enforced conjunction / RESTRICTIVE, not a permissive OR). An INSERT under an absent `mosaic.workspace_id` GUC **ERRORS** (fail-closed on NULL); a same-workspace INSERT with a valid status **succeeds** (predicate composes with F1/N-3, does not over-deny).
|
||||
|
||||
### 2.5 Invariant / key / tenancy field-set (F7 — closure)
|
||||
|
||||
- **`packages/db/src/sink-rbac/invariant-set.ts`** defines the invariant field-set = every sink column NOT in the explicitly-mutable set: `{ natural-key columns (workspace_id, id), status, creation/tenancy columns, immutable metadata }`. The complement (title, tags-normalized, due_at, rank, …) is the mutable set the UPSERT `DO UPDATE` touches.
|
||||
- **[F7 FIX] `UPDATE` on key/tenancy/invariant columns (`workspace_id`, `id`, tenancy columns) is REVOKED from ALL rungs — including User and Orchestrator.** A re-key would otherwise **launder status** (relocate a row into a new identity to escape the status invariant). Column-level `UPDATE` grants for every rung exclude these columns.
|
||||
- **Grant/data-plane disjointness (RC19-B1-03):** grants are declarative + physically separate from the data plane, so a grant/policy mutation writes **zero** `tasks` rows. A test applies an arbitrary policy/grant delta → asserts **zero byte delta** on any `tasks` row (closes RC19-B1-03 by construction).
|
||||
|
||||
### 2.6 Per-writer attribution → metadata column (Q1 audit channel)
|
||||
|
||||
- Per-writer attribution (which federated writer inside a tier acted) is an **AUDIT** concern, not an authority concern. It is captured as an **app-supplied `actor_id` metadata column on `task_events`** (append-only, already runtime `INSERT/SELECT`-only per frozen §4 L178). KBN-100 owns the column; the application supplies the verified federated-writer identity as **data**.
|
||||
- It is **never** a DB credential, role, or RLS predicate key, and (per F6) never a `userOverride`/authority key (the override is keyed on task+tier, not writer). This keeps "which writer acted" auditable **without** a per-writer DB topology — i.e. without hitting the Q1 HALT trigger.
|
||||
|
||||
### 2.7 No-status-write via trigger / view / rule (F3 BLOCKING + N-1 + N-2)
|
||||
|
||||
Column-level `UPDATE(status)` alone does not close every indirect status-write path. Three complementary invariants (homed in KBN-100 producer; enumerated by -10's `invariant-set.ts`/no-status enumeration; certified deployed by KBN-101-09):
|
||||
|
||||
- **F3 — no status-normalizing trigger on `tasks`:** no trigger on `tasks` writes/normalizes `status` outside the sanctioned `transitionStatus()` path. The transition path is the **only** status writer.
|
||||
- **[N-2 FIX + NB-1 + continuous-scan ADOPTED] Complete no-status-write enumeration (F3 completeness):** the cert asserts, in addition to F3: **no AFTER trigger** on `tasks` executing `UPDATE tasks SET status`; **no `SECURITY DEFINER` function** that writes `tasks.status` (frozen §4 L176 already forbids `SECURITY DEFINER` unless a separately reviewed exception — cited as the backstop); **no trigger on a RELATED runtime-writable table** (`task_events` and the other §4 L178 relations) that writes `tasks.status`; and **no `CREATE RULE`** on `tasks`. **[NB-1]** The enumeration is **extended to the `task_status_write_override` relation**: no status-writing trigger / RULE / `SECURITY DEFINER` function on the override relation may write `tasks.status` (the override table is runtime-writable by the User rung, so it is exactly the class §2.7's related-table clause must cover). The enumeration **scans function BODIES** (`pg_proc.prosrc` / dependency graph) for `tasks.status` writes, not merely `pg_trigger` rows. **[Gate A-delta-2 obs #2 — ADOPTED, not just recommended]** this enumeration is now a **continuous -06 CI catalog scan run on EVERY migration** (promoted from the v3 one-shot -09 recommendation), so a _later_ migration that adds a status-writing trigger/function/rule/view on `tasks` **or** on `task_status_write_override` **fails CI**, not only the one-time -09 cert. The -09 deployed cert still records the final deployed proof; the -06 scan is the standing gate. **[v6 — N1]** the same continuous -06 scan family carries the **strengthened `pg_policy.polpermissive` composition lint** (§1.4 item 3-bis / §2.4 clause 5): on every migration it counts permissive policies **grouped by effective role — including `PUBLIC` and role inheritance** (a `TO PUBLIC`/inherited permissive applicable to a writable command counts alongside that rung's status-pin) and flags an **internally-disjunctive single `WITH CHECK`** (`status_pin OR workspace_bind` within one policy). This lint is **supplementary**: it backstops the catalog shape but is **not sufficient** — the behavioral positive-controls (§2.4 clause 5 / §2.9) remain the PRIMARY proof that the composition is single-compound/`AS RESTRICTIVE` rather than an OR of permissive policies.
|
||||
- **[N-1 FIX] no status-write via VIEW or RULE:** forbid any `VIEW` or `RULE` on `tasks` that yields a status-write path. An owner-owned view runs with the owner's privileges (`security_invoker = false` by default), side-stepping rung grants; an `ON INSERT/UPDATE DO INSTEAD` rule rewrites a write onto `tasks` similarly. The `invariant-set.ts` enumeration **forbids such views/rules OR mandates `security_invoker = true` + ZERO runtime grants** on any view over `tasks`. A negative test proves a rung cannot mutate `tasks.status` through any view/rule.
|
||||
|
||||
### 2.8 B2 — SCM_RIGHTS credential-acquisition + non-dumpable re-verify (F5 hardened — unchanged from v2)
|
||||
|
||||
- **`packages/db/src/credential-handoff/scm-rights.ts`** — the privileged process (the **-03 migrator launch wrapper**) opens the privileged fd (attestation signing key, §5 L191) **before** the UID drop and passes the **descriptor itself** over a Unix-domain socket via an `SCM_RIGHTS` ancillary message. The receiver **never** re-opens `/proc/self/fd/N` (the kernel re-checks permission and returns EACCES after the drop — the FD5 failure). `O_CLOEXEC` managed explicitly on both ends.
|
||||
- **[F5 FIX] Socket authentication:** the transfer socket **MUST** be a `socketpair()` created **pre-fork** (no filesystem socket) **OR** a filesystem socket in a `0700` directory verified with **`SO_PEERCRED`** (assert peer UID/GID/PID). The receiver sets **`MSG_CMSG_CLOEXEC`** on `recvmsg`. An unauthenticated/anonymous peer is rejected fail-closed.
|
||||
- **`packages/db/src/credential-handoff/process-hardening.ts`** — after **every** credential/UID transition, re-assert `prctl(PR_SET_DUMPABLE, 0)` and **verify** `prctl(PR_GET_DUMPABLE) == 0`; treat `dumpable != 0` as a fail-closed abort.
|
||||
- **Consumers:** KBN-101-03 migrator wrapper (drop to `10003:10003`), KBN-101-02 importer (privileged-fd case, drop to `10002:10002`). The runtime rung-connection selection is a **DB-role choice, NOT an OS-UID transition** — no new setuid surface, no dumpability handling, reinforcing the no-per-user / no-new-privilege-drop property.
|
||||
|
||||
### 2.9 [B-1 FIX] Sink-resident, task/tier-scoped User-override table (the User deny is now enforced IN PostgreSQL)
|
||||
|
||||
**Problem (Gate B-delta B-1):** in v2 the evaluator resolved the User-override in-app _before_ selecting a connection, so a compromised/buggy Gateway on the Orchestrator pool could bypass a User deny — contradicting v2 §1.4/Q2, which mandates that "an Orchestrator write vetoed by a User-override deny must ERROR at the sink." App-resolution is not sink-enforcement.
|
||||
|
||||
**Fix (option a — DB-resident override, RLS-forced):**
|
||||
|
||||
- **New relation `mosaic.task_status_write_override`** (owned/created by the KBN-100 producer, exactly like `tasks`), columns: `workspace_id`, `task_id`, `tier` (enum `{orchestrator}` for v1 — the only vetoable sub-god tier; extensible to a future 4th rung), `active boolean`, `created_by_actor_id` (audit metadata, NOT an authority key), timestamps. **Primary/unique key `(workspace_id, task_id, tier)`** — **task-identity + tier ONLY; NO per-federated-writer column is part of the key or an authority input** (preserves F6 + the Q1 HALT boundary: every Orchestrator-tier writer is vetoed identically for a given task).
|
||||
- **Who may WRITE it (sink-enforced):** `INSERT/UPDATE/DELETE` on the override table is granted **ONLY** to `mosaic_runtime_user_capability` (the User god rung), with the table under `FORCE RLS` and a `WITH CHECK` policy binding the row's `workspace_id` to the session workspace predicate. **[B2-INSERT parity]** that `WITH CHECK` workspace binding applies to the override table's **INSERT** path too (`workspace_id = current_setting('mosaic.workspace_id', true)::uuid`, fail-closed on NULL), so the override table's write path carries the identical symmetric tenant isolation as `tasks` — a User-rung INSERT of a veto row for a foreign workspace, or under an absent GUC, **ERRORS at the sink**. **[v5 COMPOSITION MANDATE parity]** the override table's write policies are subject to the SAME composition rule as `tasks` (§1.4 item 3-bis / §2.4 clause 5): the workspace-bind (and any status/authority conjunct) MUST be realized as a **single compound `WITH CHECK`** OR **`AS RESTRICTIVE`** — **never as multiple _permissive_ policies intended to AND** (which would OR-admit a foreign-workspace veto row). The Orchestrator and base rungs have **no** write privilege — a non-User attempt to author/clear a veto **ERRORS at the sink**. Thus only the User rung can raise or lift a veto.
|
||||
- **Who READS it:** all three rung capabilities get **`SELECT`** on the override table (workspace-scoped `USING`), because the `tasks` write policy subquery evaluates as the current (querying) rung role. The read is confined to the querying connection's workspace by the override table's own `USING` predicate.
|
||||
- **How it vetoes (the subquery):** every `tasks` **write policy** for the **Orchestrator rung** (`INSERT … WITH CHECK`, `UPDATE … WITH CHECK`, and the transition `WITH CHECK`) is extended with:
|
||||
|
||||
```sql
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM mosaic.task_status_write_override o
|
||||
WHERE o.workspace_id = tasks.workspace_id
|
||||
AND o.task_id = tasks.id
|
||||
AND o.tier = 'orchestrator'
|
||||
AND o.active
|
||||
)
|
||||
```
|
||||
|
||||
So an Orchestrator-tier write to a task carrying an active User deny is **REJECTED in PostgreSQL**, regardless of app-layer behavior. **[v5 COMPOSITION MANDATE]** this `NOT EXISTS(<override>)` subquery MUST be a **conjunct of the Orchestrator rung's single compound `WITH CHECK`** (alongside the status-pin and the workspace-bind), OR authored **`AS RESTRICTIVE`** — it may **NOT** be a separate _permissive_ policy intended to AND, because permissive policies OR and an Orchestrator write satisfying the status-pin permissive policy would then be OR-admitted despite an active deny (bypassing B-1). The User rung's own `tasks` policies do **not** subquery the override (User = god; the User authored the deny and overrides it). The base rung already cannot write status.
|
||||
|
||||
- **Per-tier NEGATIVE test (real PostgreSQL, red-first; v5-HARDENED):** insert an active override row for `(W, T, orchestrator)` via the User rung; then an Orchestrator-rung write on `(W, T)` **using a transition the Orchestrator pin OTHERWISE ACCEPTS** (a legal non-terminal `UPDATE`/transition that would succeed absent the deny row, and under the correct workspace GUC) **must ERROR at the sink** — so the rejection **isolates the override subquery** (not a status/workspace rejection). **Positive-control:** removing the override subquery **alone** flips this negative to a **PASS**, proving the _override_ — not the status-pin or workspace-bind — is doing the rejecting (and proving the subquery is a single-enforced conjunct / RESTRICTIVE, not an OR of separate permissive policies). Clearing the row (`active=false`, User rung only) re-permits the Orchestrator write; a non-User attempt to write the override table **ERRORS**. This proves the User-override is **sink-enforced** (satisfies v2 Q2) and stays within Mos Q1 tier-level (task/tier-scoped, not per-writer).
|
||||
|
||||
**[NB-2 — compromise-resistance scope of B-1.]** B-1 sink-enforces the veto against the **Orchestrator-rung path** — an Orchestrator-pool write to a vetoed task ERRORS in PostgreSQL regardless of app behavior. It does **NOT** claim to stop a Gateway compromised badly enough to hold the **User** pool: User = god and holds the sole write on the override table, so a compromised-User-pool Gateway can itself clear the veto. That is the **accepted co-resident-pool residual** (Mos Q1, NO HALT), not a defended boundary. The v3 §0 framing "a compromised Gateway cannot bypass a User deny" is corrected to "an Orchestrator-**path** write cannot bypass a User deny at the sink." Defending against a compromised User-pool Gateway would require per-federated-user DB credentials = the HALT boundary, which Mos Q1 explicitly declines.
|
||||
|
||||
**HALT note:** because the key is `(workspace_id, task_id, tier)` and never `writer`, this fix does **not** give two same-tier writers different authority — it gives the _whole_ Orchestrator tier the _same_ per-task veto. The B2-INSERT workspace predicate is likewise applied uniformly to every writer in a tier. NO HALT.
|
||||
|
||||
---
|
||||
|
||||
## 3. Findings-closure map (each → fail-closed, red-first testable)
|
||||
|
||||
| Finding | Closure | Fail-closed enforcement point | Test (red-first, real PostgreSQL / real UID drop) |
|
||||
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **RC19-B1-01** raw sink has no enforceable authority boundary | Declarative deny-by-default RBAC (§2.1) → fixed rung → per-rung DB credential; PG enforces via revoked base privileges + `FORCE RLS` deny-all + per-rung `WITH CHECK`. | At the sink: effective DB role lacks the privilege; mis-authorized write errors in PG. | -00: rung roles created, safe attrs, non-owner. KBN-100: base `UPDATE(status)`/`DELETE` denied, User/Orch allowed. -01: checkout identity==rung + safe-attr. |
|
||||
| **RC19-B1-02** DELETE+default-INSERT resets status | Status-preserving UPSERT on `(workspace_id,id)`; `status` excluded from `DO UPDATE`; `DELETE` revoked all rungs; status only via authorized transition. | At the sink: DELETE fails closed; refresh cannot touch `status`. | KBN-100 + -09: refresh preserves `status` byte-stable; DELETE denied; transition-only status change. |
|
||||
| **RC19-B1-03** grant change breaks non-status compatibility | Invariant field-set (§2.5); grants declarative + physically separate → grant mutation writes zero `tasks` rows. | Structural: data plane and grant plane disjoint. | Apply arbitrary policy/grant delta → **zero byte delta** on all `tasks` rows + invariant-set membership test. |
|
||||
| **RC19-B2-01** FD5 `/proc/self/fd` EACCES after UID drop | SCM_RIGHTS descriptor-passing (§2.8); never re-open `/proc/self/fd/N` post-drop; explicit `O_CLOEXEC`; authenticated socket (F5). | Post-drop process receives a live fd; no privileged re-open. | Drop UID → `/proc/self/fd` re-open EACCES/absent vs SCM_RIGHTS receive succeeds; unauthenticated peer rejected. |
|
||||
| **RC19-B2-02** dumpability resets after credential transition | Re-assert + **verify** `PR_SET_DUMPABLE=0` after every transition; fail closed if `PR_GET_DUMPABLE != 0`. | Process aborts if dumpable ≠ 0 post-transition. | After simulated setuid: assert `PR_GET_DUMPABLE == 0`; inject reset → fail-closed abort. |
|
||||
| **F1 (BLOCKING)** INSERT-time status unconstrained for base rung | `GRANT INSERT (non-status list)` → status DEFAULT for base; RLS `INSERT … WITH CHECK` pins new-row status to initial; `INSERT(status)` only User/Orch (§2.4). | At the sink: base status-on-insert denied by column-privilege and/or RLS `WITH CHECK`. | **Red-first:** base `INSERT … status='done'` ERRORS; base INSERT omitting status → DEFAULT; User/Orch initial-status INSERT succeeds. |
|
||||
| **F2 (BLOCKING)** rung LOGIN roles lack declared/verified safe attributes | Declare both rung roles `NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS` + non-owner (§1.3); extend L222 checkout assertion (§2.2). | Bootstrap catalog + every checkout: fail closed on SUPERUSER/BYPASSRLS/ownership. | -00 catalog asserts attrs + non-ownership; -01 injects unsafe attr/ownership → `DATABASE_ROLE_UNSAFE`. |
|
||||
| **F3 (BLOCKING)** BEFORE-trigger status bypass | Invariant: no trigger on `tasks` writes/normalizes `status` outside `transitionStatus()` (§2.7); homed in KBN-100 producer. | Producer DDL admits no status-normalizing trigger; transition path is only status writer. | KBN-100 static: enumerate triggers on `tasks`, none assign status. **-09 deployed:** base UPDATE of a permitted col cannot change status via any trigger. |
|
||||
| **F4 (non-blocking)** no frozen `tasks(workspace_id,id)` unique key | Bind `tasks (workspace_id,id)` UNIQUE as explicit KBN-100 req; else re-bind UPSERT to `(id)` (§2.3). | `ON CONFLICT` resolves to a real unique index; else hard planning error (fails loud). | KBN-100 migration: unique key exists; UPSERT plans/executes; missing-key variant fails loud. |
|
||||
| **F5 (non-blocking)** SCM_RIGHTS socket unauthenticated | `socketpair()` pre-fork **or** `SO_PEERCRED` + `0700` dir; `MSG_CMSG_CLOEXEC` on `recvmsg` (§2.8). | Fd transfer only over an authenticated peer; received fd close-on-exec. | Unauthenticated/wrong-peer rejected; `SO_PEERCRED` mismatch fails closed; `MSG_CMSG_CLOEXEC` asserted. |
|
||||
| **F6 (non-blocking)** `userOverride` scope | Schema constrains override to `(workspace_id,task_id,tier)`; **forbids** per-writer key (§2.1); realized as the sink-resident override table (§2.9). Attribution → metadata (§2.6). | Fail-closed at policy load: a per-writer authority key **fails schema validation**. | Schema test: per-writer override/authority key **rejected**; tier/task-scoped accepted. |
|
||||
| **F7 (non-blocking)** re-key launders status | Revoke `UPDATE(workspace_id,id,tenancy)` from **all** rungs incl. User/Orch (§2.5). | At the sink: no rung may UPDATE key/tenancy columns. | Each rung (incl. User) `UPDATE workspace_id`/`id` **ERRORS**. |
|
||||
| **B-1 (BLOCKING; composition PINNED in v5)** User-override was app-enforced, not sink-enforced | Sink-resident `task_status_write_override` table `(workspace_id,task_id,tier)`, User-rung-write-only; the `NOT EXISTS(<active deny>)` subquery is a **conjunct of the Orchestrator rung's single compound `WITH CHECK`** (or authored **`AS RESTRICTIVE`**), **never a separate permissive policy** — else an Orchestrator write satisfying the status-pin permissive policy is OR-admitted despite the deny (§2.9 / §1.4 item 3-bis). | At the sink: an active User deny → Orchestrator write REJECTED in PG (single enforced check, not OR-bypassable); non-User cannot author the veto. | **Per-tier NEGATIVE (real PG; v5-hardened):** User inserts deny row → Orchestrator write **using a transition the pin OTHERWISE ACCEPTS** **ERRORS** (isolates the override). **Positive-control:** dropping the override subquery alone flips it to PASS (proves single-conjunction/RESTRICTIVE, not permissive-OR). Clear row → re-permitted; Orch/base write to override table **ERRORS**. |
|
||||
| **B-2 (BLOCKING)** FORCE + `WITH CHECK`-only bricks all reads (SELECT returns 0 rows for every rung and owner) | Add per-rung `SELECT`/`UPDATE` `USING` policies, **workspace-scoped** via `current_setting('mosaic.workspace_id')`; forbid `USING(true)` (§1.4/§2.2). Homes tenant isolation at the sink. | At the sink: reads succeed only for the session's workspace; wrong/absent predicate → zero rows (no cross-tenant leak). | Post-activation: each rung `SELECT`/UPSERT-probe within its workspace **succeeds**; a `SELECT` under a foreign/absent `mosaic.workspace_id` returns **zero** rows; a `USING(true)` variant **fails the isolation negative**. |
|
||||
| **B-3 (BLOCKING)** owner/backfill write path under FORCE RLS | Backfill/repair DML runs FIRST (RLS disabled); `ENABLE`/`FORCE RLS` is the **TERMINAL** migration step; **no standing schema_owner ALLOW write policy** after activation; break-glass (`mosaic_extension_owner` §4.1 L146 + external bootstrap actor §9 L266) named in -09 cert (§1.4). | Migration ordering: backfill completes before FORCE binds; post-activation owner write has no policy → fails closed. | KBN-100 migration test: backfill INSERTs succeed pre-FORCE; **post-activation `mosaic_schema_owner` `tasks` write ERRORS under FORCE**; no runtime rung is superuser/BYPASSRLS; -09 names the two break-glass actors as the accepted boundary. |
|
||||
| **N-1 (non-blocking)** VIEW/RULE status-write path | Forbid views/rules on `tasks` yielding a status write, OR mandate `security_invoker=true` + zero runtime grants; enumerated by `invariant-set.ts` (§2.7). | Enumeration admits no status-writable view/rule; any view over `tasks` is invoker-rights + ungranted. | Negative: a rung cannot mutate `tasks.status` through any view/rule; enumeration fails an owner-rights view with runtime grants. |
|
||||
| **N-2 (non-blocking)** F3 enumeration incomplete | Extend enumeration: no AFTER trigger `UPDATE tasks SET status`; no `SECURITY DEFINER` fn writing `tasks.status` (L176 backstop); no trigger on related runtime-writable table (`task_events`, L178) writing `tasks.status`; no `CREATE RULE`; scan function **BODIES** (§2.7). One-shot -09 cert; recommend continuous -06 CI scan. | Producer/-09 cert admits no indirect status-write path. | Catalog scan of `pg_proc.prosrc`/triggers/rules asserts none write `tasks.status`; deployed behavioral proof; (recommended) -06 CI re-scan per migration. |
|
||||
| **N-3 (non-blocking)** Orchestrator can INSERT terminal status | Orchestrator `INSERT(status)` `WITH CHECK` pinned to non-terminal initial status; only User may INSERT arbitrary/terminal (§2.4). | At the sink: Orchestrator terminal-status INSERT rejected by `WITH CHECK`. | Orch `INSERT … status='done'` **ERRORS**; Orch `status='backlog'` succeeds; User `status='done'` succeeds. |
|
||||
| **N-4 (non-blocking)** transition-graph legality not DB-enforced (`done→backlog` reset by another name) | **DECISION (documented, §2.10):** edge-legality is **app-enforced** within near-god/god authority; the sink closes RC19-B1-02 **for base only** (DELETE-denial + no base status write). No status-writing trigger is added (would collide with F3/N-2); a validation-only guard is deferred. | Documented boundary: User/Orch hold near-god/god authority; illegal edges are an app-layer concern, not a sink guarantee. | -09 records the decision + asserts base cannot reset status at the sink; app-layer transition-graph tests are owned by the Gateway command lane (out of KBN-101 scope). |
|
||||
| **N-5 / AD-2 (non-blocking)** import direction ambiguity | `-01` provisions generic pools only; rung→pool SELECTION lives in `-10`'s `connection-selection.ts`, consumed by `-02/-03/-05`; **never imported by `-01`** → `10→01`, acyclic (§2.2). | Structural: no `01→10` edge exists. | Static import test: `-01` has no import of `sink-rbac/connection-selection`; `-02/-03/-05` do; graph acyclic. |
|
||||
| **AD-1 (completeness)** frozen §4 L176 CONNECT cert reds on the 2 new logins | Amend the L176 allowlist to admit `mosaic_runtime_user`/`mosaic_runtime_orchestrator` + assert their `TEMPORARY` denial; named in -00 row/rc.20 (§1.3). | -00 cert: CONNECT allowlist includes the 2 new logins; all 3 runtime rungs `TEMPORARY`-denied. | -00 cert asserts CONNECT granted to the 2 new logins and no runtime rung retains `TEMPORARY`; an un-amended allowlist REDS. |
|
||||
| **B2-INSERT (BLOCKING; composition PINNED in v5)** tenant isolation not enforced on the INSERT write path (v3 bound workspace only in `SELECT`/`UPDATE` `USING`; PostgreSQL does not apply `USING` to INSERT, and the explicit status `WITH CHECK` suppresses `USING` substitution → `workspace_id` unbound on INSERT → any rung can INSERT/UPSERT into a foreign workspace). **[v5 delta-3]** the v4 justification ("all `WITH CHECK` AND") was FALSE — multiple permissive policies OR — so the workspace conjunct must be pinned into a _single compound check or `AS RESTRICTIVE`_, else the bind is bypassable. | Include `workspace_id = current_setting('mosaic.workspace_id', true)::uuid` as a **conjunct of the single compound INSERT `WITH CHECK`** (with the F1/N-3 status-pin and, Orchestrator, the B-1 override subquery) **OR** author it **`AS RESTRICTIVE`**, on `tasks` (and the UPSERT INSERT arm) + symmetric on `task_status_write_override`. **Multiple _permissive_ policies intended to AND are EXPLICITLY FORBIDDEN** (they OR). Corrected semantics + realization mandate in §1.4 item 3-bis / §2.4 clauses 4–5 / §2.9. | At the sink: an INSERT/UPSERT with a foreign `workspace_id` fails the single enforced check; a NULL GUC → NULL predicate → INSERT ERRORS (fail-closed). Write-side tenant isolation homed at the sink and no longer OR-bypassable. | **Red-first (real PG; v5-hardened):** the cross-workspace negative uses an **otherwise-valid row** (base + `status='backlog'` + no override, differing only in `workspace_id=<foreign W2>`) so the **ERROR** isolates the workspace bind; direct and via status-upsert INSERT arm. **Positive-control:** dropping the workspace conjunct alone flips it to PASS (proves single-conjunction/RESTRICTIVE, not permissive-OR). NULL-`mosaic.workspace_id` INSERT **ERRORS**; same-workspace valid-status INSERT **succeeds** (no over-deny); override-table foreign-workspace INSERT **ERRORS**. |
|
||||
| **NB-1 (non-blocking)** override relation absent from the no-status enumeration | Extend the F3/N-1/N-2 no-status-write enumeration **and the -09 cert / continuous -06 scan** to `task_status_write_override`: no status-writing trigger / RULE / `SECURITY DEFINER` function on the override relation writes `tasks.status` (§2.7). | Enumeration/CI admits no indirect status-write path via the override relation. | Catalog + fn-body scan asserts no trigger/rule/SECURITY-DEFINER on `task_status_write_override` writes `tasks.status`; a planted one REDS the -06 scan. |
|
||||
| **NB-2 (non-blocking)** overstated B-1 compromise claim | Scope §0/§2.9 wording: B-1 sink-enforces the veto against the **Orchestrator-rung path**; a compromised **User**-pool Gateway (User=god) can clear a veto — accepted Mos Q1 co-resident residual, NO HALT (§0/§2.9). | Documented boundary: sink-enforcement is Orchestrator-path-scoped; the User-pool residual is accepted, not defended. | Doc/claim assertion in -09 cert notes: the override-veto negative proves the **Orchestrator-path** ERROR; the framing no longer claims User-pool compromise resistance. |
|
||||
| **NB-3 (non-blocking)** overstated B-2 compromise claim | Scope §0/§2.2 wording: the workspace GUC is app-set with no per-tenant DB check → B-2 + B2-INSERT protect a **buggy** Gateway (omitted predicate → fail-closed), NOT a **compromised** one forging the GUC (accepted residual). The B2-INSERT unbound-INSERT closure remains a real fail-closed win (§0/§2.2). | Documented boundary: fail-closed on omission/NULL; forged-GUC cross-tenant is the accepted per-user residual. | Cross-workspace read → zero rows; NULL/foreign-GUC INSERT ERRORS (buggy-Gateway closure proven); doc states forged-GUC is out of scope (HALT boundary). |
|
||||
| **NB-4 (non-blocking; v5 states transitive)** owner-toggle-FORCE containment mis-stated as incapability | Name `mosaic_schema_owner` (reachable via `mosaic_migrator` `SET ROLE`) in the -09 break-glass enum as **contained-but-CAPABLE**: an owner can `ALTER TABLE tasks DISABLE/NO FORCE RLS` then write regardless of NOSUPERUSER/NOBYPASSRLS → containment = **transitive non-reachability from the 3 runtime LOGIN rungs**, asserted explicitly (§1.4). | -09 cert: owner toggle-then-write is real but confined to the migrator/owner phase; runtime rungs cannot reach `mosaic_schema_owner`/`mosaic_migrator` through any inherited (capability-role) path. | **[NB-4-transitive]** -09 asserts non-reachability as **TRANSITIVE membership** — `NOT pg_has_role(<rung>, 'mosaic_schema_owner', 'USAGE')` / `NOT has_privs_of_role(...)` recursion, NOT merely the absence of a direct `pg_auth_members` row (ownership check returns true for an _inherited_ membership through a capability role). Frozen §6 L222 "assumability" plausibly already covers this; "transitive membership" is stated explicitly. **[v6 — F-NB4]** the assertion additionally **conjoins the `MEMBER` variant** — `NOT pg_has_role(<rung>, 'mosaic_schema_owner', 'MEMBER')` **AND** `NOT pg_has_role(<rung>, 'mosaic_migrator', 'MEMBER')` for each of the 3 runtime rungs — because a `WITH INHERIT FALSE, SET TRUE` membership yields `has_privs_of_role = FALSE` yet still permits `SET ROLE … ; ALTER TABLE … NO FORCE`; the `MEMBER` privilege captures SET-ROLE reachability regardless of `INHERIT` (the intended graph has each rung `SET FALSE` to only its capability role, so this is assertion-completeness, not a live hole). `mosaic_schema_owner` is `NOLOGIN`, reachable only via `mosaic_migrator` `SET ROLE`; the 3 runtime rungs are transitive non-members and non-assumers (catalog + L222 checkout). |
|
||||
| **NB-5 (non-blocking)** 2 new capability roles lack baseline runtime grants → fail frozen §6 L222 verify | Grant `mosaic_runtime_user_capability` / `mosaic_runtime_orchestrator_capability` the frozen §4 L176 baseline runtime grants (`USAGE ON SCHEMA mosaic`; `USAGE ON SCHEMA drizzle` + `SELECT` on the two ledger relations; relevant sequence `USAGE`/`SELECT`) at -00 (§1.1/§1.3). | Bootstrap: the 2 new rungs hold the baseline grants → they connect/operate and pass the L222 runtime verify. | -00 catalog asserts each new capability role holds `USAGE ON mosaic`, ledger `SELECT`, sequence grants; an omission → L222 runtime verify fails closed (missing inherited grant). |
|
||||
| **Q2 per-tier deny-by-default (red-team requirement; v5 composition-pinned)** | Per-rung **single compound `WITH CHECK`** (status-pin + B2-INSERT workspace bind + Orchestrator override) **or `AS RESTRICTIVE`** — **never multi-permissive-intended-to-AND** — plus `USING` + `FORCE RLS`; deny-all default (§1.4 item 3-bis). | At the sink, every tier: unauthorized write-source rejected even for owner/maintenance (`FORCE`); tenant/override binds not OR-bypassable. | **NEGATIVE per tier (v5-hardened):** base terminal-status INSERT/transition ERROR; **Orchestrator write under User-override deny, using an otherwise-accepted transition, ERROR (+ positive-control: drop override subquery → PASS)**; **cross-workspace INSERT/UPSERT with an otherwise-valid status ERROR (+ positive-control: drop workspace conjunct → PASS)**; NULL-GUC INSERT ERROR; owner/maintenance post-activation write ERROR (`FORCE` proven); cross-workspace read → zero rows; **same-workspace valid-status write SUCCEEDS per writable command (INSERT, transition/UPDATE, SELECT) — the [v6 — N2] ≥1-permissive-per-writable(rung,command) option-(b) vacuous-deny guard**; `pg_policy.polpermissive` check confirms single-compound-or-RESTRICTIVE, **[v6 — N1] grouped by effective role incl. `PUBLIC`/inheritance and rejecting an internally-disjunctive single `WITH CHECK` — supplementary to, never a substitute for, the behavioral positive-controls (PRIMARY proof)**. |
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation card DAG (owner + disjoint manifest, red-first at Gate A/B)
|
||||
|
||||
```
|
||||
contract(amended) ─► KBN-101-00 (rung-ROLES + capability roles, safe attrs [F2],
|
||||
│ deny-default base, CONNECT + [AD-1] L176 allowlist
|
||||
│ amend (2 new logins CONNECT, TEMPORARY-denied);
|
||||
│ bootstrap catalog tests. NO tasks/override DDL [GA-1])
|
||||
│
|
||||
KBN-101-01 ──┤ (provisions 3 generic rung pools; sets mosaic.workspace_id
|
||||
│ session predicate [B-2]; checkout identity==rung AND
|
||||
│ safe-attribute/non-owner verify [F2, extended L222].
|
||||
│ Does NOT import -10 [AD-2/N-5])
|
||||
│
|
||||
KBN-101-10 ──┘ depends 00,01:
|
||||
packages/db/src/sink-rbac/** (policy+schema[F6]+evaluator+ladder+
|
||||
connection-selection[consumes 01; 10→01, N-5]+
|
||||
status-upsert[F4]+invariant-set[F7]+
|
||||
no-status trigger/view/rule enum[N-1,N-2])
|
||||
packages/db/src/credential-handoff/** (scm-rights[F5] + process-hardening +specs)
|
||||
│
|
||||
┌─────────────┬────────┼──────────────────────┬──────────────────────────────┐
|
||||
KBN-101-03 KBN-101-05 KBN-101-02 KBN-100 (producer lane) KBN-101-09 (evidence-only)
|
||||
(dep 00,01,10) (dep 00,03, (dep 01,03,10) SPEC target of -10: (dep KBN-100,08)
|
||||
migrator 10) importer fd + homes tasks + override DDL DEPLOYED cert:
|
||||
wrapper: renderer: sink adapter uses AFTER create: real rung INSERT/
|
||||
SCM_RIGHTS 3 rung DSN status-upsert - rung column-grant matrix transition success;
|
||||
SEND + dumpable secret (drop 10002) [F1,F7,N-3] override-veto [B-1];
|
||||
re-assert mounts; - RLS WITH CHECK(+override) cross-workspace read
|
||||
(drop 10003) Gateway + USING workspace pred zero-rows [B-2];
|
||||
[GA-4(ii): Dockerfile; [Q2,B-1,B-2, conds a&b] backfill-then-FORCE
|
||||
docker/db- NO SCM_RIGHTS - task_status_write_override ordering + owner-write
|
||||
migrator. (DB-role, no table [B-1] denial [B-3] + break-
|
||||
Dockerfile] UID drop) - tasks(workspace_id,id) UNIQUE glass named;
|
||||
│ [F4] base status
|
||||
KBN-101-06 (dep 02,03,05,07,10): inventory += KBN-101-10; INSERT/UPDATE/DELETE
|
||||
overlap/ownerless/path-existence green; denial; refresh
|
||||
matrix += rung selection + SCM_RIGHTS[F5] + status-preservation;
|
||||
per-tier RLS negatives + override-veto[B-1] + no-status trigger/
|
||||
cross-workspace read[B-2] + backfill-order[B-3] view/rule [F3,N-1,N-2];
|
||||
+ ADOPTED continuous no-status scan[N-2,NB-1] grant-delta zero-byte;
|
||||
+ cross-workspace/NULL-GUC INSERT neg[B2-INSERT]
|
||||
│ dumpable=0 evidence
|
||||
KBN-101-08 (dep 00…07,10): foundation + atomic activation cert
|
||||
```
|
||||
|
||||
- **Gate A (re-review):** every element traces to a manifest owner; disjointness/overlap/ownerless proven against amended §7; the SPEC-vs-implement split for `tasks`+override sink DDL matches the frozen §4 L178 precedent (producer = KBN-100, applied **after** table creation — GA-1); dep edges acyclic with `10→01` import direction (AD-2/N-5); the L176 CONNECT amendment named (AD-1).
|
||||
- **Gate B (red-team):** each B1/B2 finding **and** F1–F7 **and** B-1/B-2/B-3 **and** B2-INSERT **and** N-1..N-4 **and** NB-1..NB-5 **and** the Q2 per-tier deny-by-default has a **red-first** test that fails on the current design and passes only with the enforcement point; sink privilege/RLS/override/tenant denials — **including cross-workspace and NULL-GUC INSERT/UPSERT [B2-INSERT]** — proven **in PostgreSQL** (not PGlite — §5 L218); SCM_RIGHTS/dumpable against a real UID drop with an authenticated socket.
|
||||
|
||||
---
|
||||
|
||||
## 5. Executability self-audit (re-run for v5)
|
||||
|
||||
**Every element has an owner in the amended manifest — nothing ownerless:**
|
||||
|
||||
| Element | Owner |
|
||||
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
|
||||
| Declarative RBAC policy artifact + schema [F6] + evaluator + ladder + no-status trigger/view/rule enum [N-1,N-2] | KBN-101-10 (`packages/db/src/sink-rbac/**`) |
|
||||
| Rung→credential connection-selection module (consumes -01 pools; `10→01` [N-5]) | KBN-101-10; pools provisioned in KBN-101-01 (`database.module.ts`) |
|
||||
| Status-preserving-UPSERT [F4] + invariant-set [F7] module | KBN-101-10 (`packages/db/src/sink-rbac/**`) |
|
||||
| SCM_RIGHTS handoff [F5] + non-dumpable assert/verify | KBN-101-10 (`packages/db/src/credential-handoff/**`) |
|
||||
| Fixed rung DB roles + safe attributes [F2] + deny-default base + CONNECT + **[AD-1] L176 allowlist amend + TEMPORARY denial** + **[NB-5] baseline L176 runtime grants (`USAGE mosaic`, ledger `SELECT`, sequence grants) on the 2 new capability roles** | KBN-101-00 (`infra/pg-bootstrap/roles.sql` + tests) — **no tasks/override DDL [GA-1]** |
|
||||
| `tasks` sink DDL: rung grants [F1,F7,N-3] + RLS `WITH CHECK`(+override) [Q2,B-1] + **INSERT `WITH CHECK` workspace bind on every rung [B2-INSERT]** + `SELECT`/`UPDATE` `USING` workspace predicate [B-2] + `FORCE RLS` terminal-after-backfill [B-3, conds a&b] + `(workspace_id,id)` UNIQUE [F4] + `actor_id` col [Q1] + no-status trigger/view/rule invariant on `tasks` **and override relation** [F3,N-1,N-2,NB-1] | **KBN-100 producer** (SPEC by -10; §4 L178 precedent; scope amended per Q2) |
|
||||
| **`task_status_write_override` table [B-1]** (create + grants + FORCE RLS + policies; User-write-only incl. **INSERT `WITH CHECK` workspace bind [B2-INSERT parity]**, all-rung-SELECT) | **KBN-100 producer** (SPEC by -10; same L178 precedent) |
|
||||
| `mosaic.workspace_id` session predicate set at checkout [B-2] | KBN-101-01 (`database.module.ts` / `connection-identity.ts`) |
|
||||
| Rung DSN secret mounts (3 fixed) | KBN-101-05 (renderer, extends runtime-URL row) |
|
||||
| Checkout identity==rung + safe-attribute/non-owner verify [F2] | KBN-101-01 (`connection-identity.ts`, extended L222) |
|
||||
| Migrator launch wrapper SCM_RIGHTS send + dumpable re-assert | KBN-101-03 (`docker/db-migrator.Dockerfile` [GA-4(ii)]) |
|
||||
| Importer privileged-fd case | KBN-101-02 |
|
||||
| Deployed enforcement certification (F3/N-1/N-2 **+ NB-1 override-relation scan**, per-tier RLS negatives, B-1 override-veto, B-2 cross-workspace read, **B2-INSERT cross-workspace/NULL-GUC INSERT negatives**, B-3 ordering + break-glass naming **incl. [NB-4] `mosaic_schema_owner` contained-but-capable / non-reachability assertion**, N-4 decision) | KBN-101-09 (evidence-only) |
|
||||
| Inventory/overlap/path-existence + command matrix (rung sel + SCM_RIGHTS + per-tier RLS + override-veto + cross-workspace read + **[B2-INSERT] cross-workspace/NULL-GUC INSERT negatives** + backfill-order + **[ADOPTED] continuous no-status scan on `tasks` + override relation**) | KBN-101-06 (fixture += -10) |
|
||||
|
||||
**Manifest closure (KBN-101-06 green):** the only NEW ownership is the two disjoint -10 globs. The B-1 override table, B-2 `USING` policies, B-3 terminal-FORCE ordering, **and the v4 B2-INSERT INSERT `WITH CHECK` workspace binding (+ its override-table parity) and the NB-1 override-relation no-status enumeration** are ALL producer DDL homed in KBN-100's already-owned `schema.ts`/`drizzle/**` (a _responsibility_ widening, not a glob change — identical treatment to the v3 `tasks` sink DDL). The **NB-5 baseline capability-role grants** amend -00's already-owned `roles.sql`/tests (like AD-1); the **continuous -06 no-status scan** lives in -06's already-owned `tools/ci/fixtures/kbn101-ddl-inventory.json` + `.woodpecker/ci.yml`; the NB-2/NB-3/NB-4 claim-scopings/enumeration are doc + -09 cert wording on already-owned paths. N-1/N-2/N-5 additions live in -10's already-declared `sink-rbac/**`. **No glob is split/shared/narrowed → overlap/ownerless/path-existence stay green (KBN-101-06 unchanged-green vs v3).**
|
||||
|
||||
**NO per-federated-user topology (HALT boundary respected — re-verified for v5):** DB-role/credential cardinality = **3** (User / Orchestrator / others), fixed and independent of the federated-user count. Federated identity → rung happens in the evaluator at authorization time. A new federated user creates **zero** roles/credentials/secrets/pools. The **B-1 override table is keyed `(workspace_id, task_id, tier)` — task+tier, never writer** — so it homes a tier-level veto, not a per-user distinction. The **B-2 read `USING` predicate AND the B2-INSERT INSERT check predicate are the SAME shared per-request session var** (`mosaic.workspace_id`), applied uniformly to whichever rung role holds the connection — tenant isolation, not writer discrimination (and, per v5, composed into a single compound check or `AS RESTRICTIVE` rather than an OR of separate permissive policies). Per-writer **attribution** is a metadata column (`task_events.actor_id`, §2.6), not a DB principal. `userOverride` forbids a per-writer authority key (F6, fail-closed at load). The NB-1..NB-5 fixes add no principal, key, or predicate that varies by writer. Enforce-at-sink is satisfied by the fixed rung roles' grants + `FORCE RLS WITH CHECK`(+override, +workspace-bound INSERT) + `USING` — it does **not** require, and this design does **not** use, a DB role/credential per federated user.
|
||||
|
||||
**Q1 same-tier / different-authority check (the ONLY HALT trigger):** no v5 change gives two SAME-TIER writers different WRITE authority. The v5 composition pin fixes only **how** the already-uniform predicates are physically composed (single compound `WITH CHECK` or `AS RESTRICTIVE`, never multi-permissive-OR) — it changes no predicate's content and introduces no writer-varying axis. F1/F2/F3/F7/N-3 tighten authority uniformly per tier; **B2-INSERT binds the same workspace predicate uniformly into every rung's INSERT check** (tenant isolation, not writer discrimination); B-1 vetoes the _entire_ Orchestrator tier per task (not a subset of Orchestrator writers); B-2 isolates by workspace uniformly; F6 forbids per-writer authority keys; attribution is audit-only. The NB-4-transitive nit only _strengthens_ the owner non-reachability assertion (transitive membership) — no authority axis. **No same-tier-different-authority requirement exists → NO HALT — an executable envelope is delivered.**
|
||||
|
||||
---
|
||||
|
||||
## 6. Flagged ambiguities in the frozen contract (for reviewer awareness)
|
||||
|
||||
1. **Sink identity of the status store.** The RBAC-guarded sink is `tasks.status`; legacy `mission_tasks.status` stays frozen read-only. Confirm no reviewer reads "raw sink" as the legacy column.
|
||||
2. **Frozen "role graph unchanged" + "no RLS" + L176 CONNECT allowlist vs. the rulings.** SHARED-CONTRACT rc.5 L82/L88-90 froze the role graph and grant/revoke-only (no RLS); frozen §4 L176 froze the CONNECT allowlist. Jason B1 + Mos OPTION A add fixed rung-roles; **Mos Q2** authorizes RLS `WITH CHECK`/`USING` + `FORCE RLS` + the override table; **AD-1** amends the L176 allowlist for the 2 new logins. The rc.20 amendment text (§1.3/§1.4) records all three as deliberate, human-authorized departures. Reviewer should confirm rc.20 is **adopted** (not treated as a violation), that KBN-100's DDL scope is amended to include RLS + the override table on `tasks`, and that the -00 row amends the L176 allowlist.
|
||||
3. **Natural-key column list for `tasks` (F4).** `tasks (workspace_id, id)` UNIQUE is asserted by the SI-001 pattern but must be **bound + created by KBN-100**. If KBN-100's `tasks` identity differs, the UPSERT natural key re-binds to `(id)` (spec-level, no HALT).
|
||||
4. **Initial/terminal status values (F1/N-3).** F1 pins base new-row status to `backlog` (§5.4 L214); N-3 pins the Orchestrator's INSERT set to non-terminal (`{backlog, ready}`) and reserves terminal (`{done, cancelled}`; `in_review` treated as non-initial) to the User rung. KBN-100 binds the exact column DEFAULT and the RLS `WITH CHECK` literals against the §3 L134 vocabulary so they agree.
|
||||
5. **Workspace session-var name + set-site (B-2 + B2-INSERT).** The `USING` predicate **and the v4 INSERT `WITH CHECK` predicate** both use `current_setting('mosaic.workspace_id', true)`. -01 sets it per request from the server-verified tenant (never a body field, §7.1). Reviewer should confirm the exact GUC name and that a transaction-local `SET LOCAL` is used so a pooled connection cannot leak a stale workspace across requests (mirrors the frozen `SET LOCAL search_path` discipline, §4 L152) — now **explicitly folded into the frozen L174 poisoned-pooled-session-reset + `SET LOCAL` restoration negative test** (Gate B 2(a), §2.2). Reviewer should also confirm the accepted scope (NB-3): fail-closed on an omitted/NULL GUC (buggy Gateway), but a _forged_ GUC to a victim tenant is the accepted per-user residual (Mos Q1), not defended.
|
||||
6. **N-4 transition-graph decision.** Edge-legality (illegal/backward transitions such as `done→backlog`) is **app-enforced** within near-god/god authority; the sink closes RC19-B1-02 for **base only** (DELETE-denial + no base status write). A validation-only sink guard is deferred to avoid colliding with the F3/N-2 no-status-trigger invariant. Reviewer should confirm this decision is acceptable (it is stated, not silently dropped).
|
||||
7. **N-2 continuous scan — now ADOPTED.** The complete no-status-write enumeration (trigger/view/rule/SECURITY DEFINER/function-body), covering `tasks` **and `task_status_write_override` (NB-1)**, is now a **continuous -06 CI catalog scan run on every migration** (promoted from the v3 one-shot -09 recommendation, per Gate A-delta-2 obs #2); the -09 deployed cert still records the final proof. Reviewer should confirm the -06 scan is owned/wired in `tools/ci/fixtures/kbn101-ddl-inventory.json` + `.woodpecker/ci.yml` (already-owned -06 paths).
|
||||
8. **KBN-101-09 is evidence-only.** B-1/B-2/B-3/F3/N-1/N-2 negative tests are _authored/owned_ by the producing implementation cards (KBN-100 producer for `tasks`/override triggers/RLS; -10/-01 for the rung path); KBN-101-09 records the **deployed** certification evidence and changes no implementation path. Confirm the test-ownership split is acceptable (mirrors the frozen -09 "evidence-only" role).
|
||||
9. **B2-INSERT / B-1 `WITH CHECK` composition — SEMANTICS CORRECTED + REALIZATION MANDATED (v5; the delta-3 fix).** v4 justified the composition with a **FALSE** PostgreSQL claim — "multiple applicable `WITH CHECK` clauses must all pass (they AND)". **That is struck.** The **correct** semantics: multiple **PERMISSIVE** policies for a command combine with **OR** (a row is admitted if it satisfies **ANY** one permissive `WITH CHECK`, ANDed with all `AS RESTRICTIVE` checks); the AND this closure needs holds **ONLY** (a) within a _single_ policy whose `WITH CHECK` is one AND-conjoined expression, or (b) across `AS RESTRICTIVE` policies. Accordingly v5 **MANDATES** (§1.4 item 3-bis / §2.4 clause 5 / §2.9) that, per rung per command, the status-pin **AND** the workspace-bind **AND** (Orchestrator) the override subquery be realized as EITHER a **single compound `WITH CHECK`** OR the status-pin-as-sole-permissive **plus** the workspace-bind/override **`AS RESTRICTIVE`**, and **EXPLICITLY FORBIDS** realizing them as multiple _permissive_ policies intended to AND (which would OR-admit a foreign-workspace INSERT satisfying only the status-pin — cross-tenant injection — and equally bypass the B-1 veto). Reviewer should confirm: (i) the corrected OR-semantics statement replaces every "all `WITH CHECK` AND" assertion; (ii) the realized policies are single-compound or `AS RESTRICTIVE` (a `pg_policy.polpermissive` catalog check backs it); (iii) the hardened red-first negatives use an _otherwise-valid_ row (cross-workspace: base + `backlog` + foreign ws; override: an otherwise-accepted Orchestrator transition) so each rejection isolates the bind/override under test, **and each carries a positive-control** (dropping the workspace conjunct — resp. the override subquery — alone flips the negative to a PASS, proving the bind/override, not the status-pin, is rejecting). Confirm the accepted scope (NB-2/NB-3) is unchanged: write-side sink tenant isolation is complete against a _buggy_ Gateway; a _compromised_ Gateway forging the GUC or holding the User pool is the accepted Mos Q1 residual.
|
||||
|
||||
---
|
||||
|
||||
_Envelope A **v6** authored from base `b0d78d86`. **v6 = v5 + N1/N2/F-NB4 non-blocking test/lint/cert hardening ONLY — NO design predicate, policy, grant, role, or table is changed** (only test/lint/cert wording). v5 CONVERGED with BOTH terminal re-gate-4 gates GO (Gate A-delta-4 `a4ce075a`; Gate B-delta-4 `a379e0de`), NO HALT; Gate B-delta-4 flagged 3 EXPLICITLY NON-BLOCKING hardening notes "for the trunk-commit record," folded here additively: **N1** — the `pg_policy.polpermissive` lint is strengthened to count permissive policies grouped by EFFECTIVE role (incl. `PUBLIC`/inheritance) and to reject an internally-disjunctive single `WITH CHECK`, stated as SUPPLEMENTARY with the behavioral positive-controls remaining the PRIMARY proof (§1.4 item 3-bis / §2.4 clause 5 / §2.7 / §3 Q2 row); **N2** — the option-(b) vacuous-deny guard is stated explicitly as the load-bearing "≥1 PERMISSIVE policy per writable (rung, command)" invariant, verified by the "same-workspace valid-status write succeeds" positive tests for INSERT / transition-UPDATE / SELECT (§1.4 item 3-bis / §2.4 clause 5 / §3 Q2 row); **F-NB4** — owner non-reachability is extended to SET-ROLE assumability by conjoining the `MEMBER` variant `NOT pg_has_role(<rung>, 'mosaic_schema_owner', 'MEMBER')` AND `NOT pg_has_role(<rung>, 'mosaic_migrator', 'MEMBER')` for each of the 3 runtime rungs in the -09 cert (§1.4 NB-4 para / §3 NB-4 row). All three are assertion/test/lint completeness on an already-BOTH-GO design; the intended role graph is already safe. The v6 body otherwise preserves the v5 text verbatim, which in turn preserved EVERY v4 closure (all v4 checks passed except one bounded item) and closed the SINGLE remaining BLOCKING finding raised identically by BOTH re-gate-3 gates (Gate A-delta-3 `aa5cbf73` CHANGES-NEEDED; Gate B-delta-3 `aa881074` 1 blocking) — the **RLS `WITH CHECK` composition semantics**: (1) **corrected semantics** — struck the FALSE "PostgreSQL requires ALL applicable `WITH CHECK` to pass / they AND" from §1.4/§2.4/§6.9 and stated the true rule (multiple PERMISSIVE policies OR; AND holds only within a single compound `WITH CHECK` or across `AS RESTRICTIVE`); (2) **mandated the realization** — per rung per command, status-pin + workspace-bind + (Orchestrator) override subquery MUST be a **single compound `WITH CHECK`** OR the status-pin-as-sole-permissive **plus** the bind/override **`AS RESTRICTIVE`**, with **multi-permissive-intended-to-AND EXPLICITLY FORBIDDEN**, applied to `tasks` **and** `task_status_write_override` (§1.4 item 3-bis, §2.4 clauses 4–5, §2.9); (3) **hardened the red-first negatives** — cross-workspace INSERT uses an otherwise-valid row (base + `backlog` + foreign ws) and the B-1 override negative uses an otherwise-accepted Orchestrator transition, so each rejection isolates the bind/override, **each with a positive-control** (dropping the workspace conjunct — resp. override subquery — alone flips the negative to PASS); plus **NB-4-transitive** (owner non-reachability stated as TRANSITIVE membership — `NOT pg_has_role(...)`/`has_privs_of_role(...)` recursion, not a direct `pg_auth_members` edge; frozen L222 "assumability" noted as plausibly already covering it). Every other v4 closure carries forward intact (3-rung ladder / 0 per-user; F1–F7; GA-1/3/4; B-1/B-2/B-3; the B2-INSERT workspace bind itself; N-1..N-5; AD-1/AD-2; NB-1..NB-5; continuous -06 no-status scan; L174 poisoned-pool `SET LOCAL` GUC fold; rc.20). Manifest disjoint (KBN-101-06 green) — all v5 changes land on already-owned KBN-100-producer / -09 / -06 paths (DDL realization detail + test wording; no glob change). Design/contract only; no SSOT file modified, no code, no PR. HALT self-check: NO HALT (fixed 3-rung ladder; override keyed task+tier not writer; workspace predicate shared per-request and uniform per tier, now composed as single-compound/`RESTRICTIVE`; attribution → metadata; no per-federated-user DB role/credential; no same-tier different-authority requirement — both delta-3 gates independently confirmed the boundary HOLDS). **v6 changes NOTHING in this HALT calculus: N1/N2/F-NB4 add only lint-scope, a vacuous-deny test invariant, and a `MEMBER` assumability assertion — zero new principal, key, predicate, grant, role, or table, and no writer-varying axis — so NO HALT is re-confirmed for v6.** Manifest remains disjoint (KBN-101-06 green): every v6 fold lands on already-owned paths — the -06 continuous scan / `pg_policy.polpermissive` lint in `tools/ci/fixtures/kbn101-ddl-inventory.json` + `.woodpecker/ci.yml`; the option-(b) positive tests in the KBN-100-producer / -09 evidence paths; the `MEMBER` assertion in the -09 cert — no glob split/shared/narrowed. NO code, NO PR, NO SSOT edit._
|
||||
@@ -0,0 +1,197 @@
|
||||
# Mission Manifest — Mosaic Native Kanban and Canonical Task SOT P0–P3
|
||||
|
||||
**Mission status:** CANON INDEPENDENTLY APPROVED; publication in progress under issue [#751](https://git.mosaicstack.dev/mosaicstack/stack/issues/751)
|
||||
**Date:** 2026-07-14
|
||||
**Human decision owner:** Jason
|
||||
**Orchestrator/publication owner:** web1 control plane (`mos-claude`; `mosaic-100` acting during Claude quota outage)
|
||||
**Execution topology:** USC web1, partitioned across collision-free GPT coder2/3/4/5 lanes
|
||||
**Canonical requirements:** [`../requirements/native-kanban-sot.md`](../requirements/native-kanban-sot.md)
|
||||
**Frozen integration contract:** `SHARED-CONTRACT.md` and `contracts/*.v1.ts`
|
||||
|
||||
## 1. Mission statement
|
||||
|
||||
Extend current `mosaicstack/stack` main into the sole native control plane for workspace-scoped project, mission, milestone, task, dependency, assignment, lease, approval, evidence, and audit state. First deliver a thin writable Kanban/List vertical slice; then add deterministic mechanical coordination and execute a one-way migration/cutover from jarvis-brain/Vikunja project/task stores.
|
||||
|
||||
Success means every user, agent, orchestrator, specialist, and UI sees and mutates the same PostgreSQL aggregate revisions through typed Gateway commands, with no writable fallback and no hidden second authority.
|
||||
|
||||
## 2. Scope boundaries
|
||||
|
||||
### In scope
|
||||
|
||||
- Current Drizzle/PostgreSQL schema extension and migrations.
|
||||
- Workspace tenancy and authorization from the first migration.
|
||||
- Projects, missions, milestones, tasks, normalized tags, dependencies, assignments, durable execution/quarantine state, links, immutable artifacts/evidence joins, outage change proposals, events, approvals, leases, checkpoints, and transactional outbox.
|
||||
- NestJS Gateway queries and explicit lifecycle commands.
|
||||
- MCP/CLI agent surfaces and generated read-only projections.
|
||||
- Thin writable Next.js Tasks Kanban/List, task detail, minimal Projects CRUD, filters, dependency readiness, ownership/lease separation, and audit timeline.
|
||||
- Non-LLM Mechanical Coordinator eligibility, proposal, approval-policy, lease/fence, heartbeat, retry, expiry, quarantine, and restart recovery.
|
||||
- Planning, Enhance, Coder, Review, SecReview, PR-Monitor, and Certifier role/gate representation.
|
||||
- One-way shadow importer, reconciliation, write freeze, final delta, cutover, rollback package, and legacy read-only stabilization.
|
||||
- Recovery-posture configuration and health-state/fail-closed contract.
|
||||
|
||||
### Out of scope
|
||||
|
||||
- Greenfield services, Prisma runtime revival, or jarvis-brain flat files as runtime storage.
|
||||
- Writable Markdown/JSON/Valkey/browser/provider fallback.
|
||||
- Gitea issue/PR replacement or generic bidirectional provider sync.
|
||||
- Calendar, email, GLPI cache, CRM, billing, time tracking, personal-brain migration.
|
||||
- LLM scheduling or scope interpretation by the Coordinator.
|
||||
- Autonomous gate waiver, certification, merge, release, deployment, or issue closure by Coordinator.
|
||||
- Merge authority for Certifier.
|
||||
- P4 full portfolio/mission designer and P5 fleet-scale policy unless separately released.
|
||||
|
||||
## 3. Fixed invariants
|
||||
|
||||
Every deployment MUST preserve all of the following:
|
||||
|
||||
1. PostgreSQL is the sole writable SOT.
|
||||
2. Drizzle on current stack main is the only persistence foundation.
|
||||
3. Mutations fail closed when DB write-health cannot be proven `healthy`.
|
||||
4. No file, Valkey, browser, queue, provider, or human note becomes a fallback writer.
|
||||
5. `TASKS.md`, `mission.json`, and every file export are generated, read-only, non-authoritative, and never import sources.
|
||||
6. Human outage notes become attributable post-recovery proposals only.
|
||||
7. Workspace is the hard tenant; Team is intra-workspace authorization.
|
||||
8. Valkey is expendable; PostgreSQL owns state, leases, fencing, audit, and outbox.
|
||||
9. Mechanical Coordinator is deterministic/non-LLM and cannot invent scope, waive gates, certify, or merge.
|
||||
10. Certifier is the final independent quality gate and has no merge authority.
|
||||
11. Mutations use idempotency and optimistic aggregate versions; worker commands also require a current fencing token.
|
||||
12. Recovery tier changes only backup/recovery posture, never authority or gate semantics.
|
||||
|
||||
## 4. Configurable recovery posture
|
||||
|
||||
Deployments select Lite, Standard, or High-assurance defaults from [`../requirements/native-kanban-sot.md`](../requirements/native-kanban-sot.md) and `contracts/recovery-posture.v1.ts`. Configurable fields are limited to:
|
||||
|
||||
- backup/base-backup cadence;
|
||||
- RPO and RTO targets;
|
||||
- PITR retention;
|
||||
- WAL archive cadence;
|
||||
- restore-test frequency;
|
||||
- break-glass drill frequency;
|
||||
- encrypted off-cluster storage.
|
||||
|
||||
High-assurance defaults are fixed reference values: RPO 15 minutes, RTO 4 hours, encrypted off-cluster WAL every 5 minutes with 35-day PITR, daily base backup, monthly restore test, and quarterly break-glass drill.
|
||||
|
||||
## 5. Canonical role map
|
||||
|
||||
```text
|
||||
User
|
||||
↓ objectives, constraints, ratified decisions
|
||||
Interaction Layer
|
||||
↓ workspace/project context; no scheduling authority
|
||||
Portfolio Orchestrator
|
||||
↓ approved mission, cross-project priority/capacity
|
||||
Project Sub-Orchestrator
|
||||
↓ decomposition, DAG, acceptance, release, routing policy, overrides
|
||||
Gateway
|
||||
↓ authenticated/authorized typed commands
|
||||
Project/Task Domain Services
|
||||
↓ transactional state + semantic event + outbox
|
||||
Mechanical Coordinator
|
||||
↓ deterministic eligibility/proposal/lease/fence/retry/quarantine
|
||||
Specialists
|
||||
Planning → Enhance → Coder → Review → conditional SecReview → remediation
|
||||
↓ complete evidence bundle
|
||||
Certifier
|
||||
↓ final pass/reject/escalate; NO merge authority
|
||||
Project Sub-Orchestrator / control plane
|
||||
↓ merge authority after all gates
|
||||
Post-merge validation
|
||||
```
|
||||
|
||||
### Authority table
|
||||
|
||||
| Role/layer | Owns | Explicitly cannot do |
|
||||
| ------------------------ | ----------------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
|
||||
| User | Objectives, constraints, Jason-owned decisions | Direct DB/file authority bypass |
|
||||
| Interaction | Conversation and context resolution | Schedule, approve, lease, certify |
|
||||
| Portfolio Orchestrator | Mission approval, cross-project priority/capacity/global holds | Implement or self-certify specialist work |
|
||||
| Project Sub-Orchestrator | Task decomposition/DAG/acceptance, release to ready, routing policy, overrides, remediation, merge go-ahead | Bypass required independent gates |
|
||||
| Gateway | Identity, tenancy, DTO validation, commands, state-machine enforcement | Accept file edits or client SQL as mutations |
|
||||
| Domain services | Transactional business invariants, semantic events/outbox | Depend on Valkey/files for committed truth |
|
||||
| Mechanical Coordinator | Eligibility, dependencies, proposal, approved routing, lease/fence, heartbeat, retry/quarantine | Invent/alter scope, waive gates, certify, merge |
|
||||
| Specialists | Bounded planning/implementation/review artifacts under a task lease | Modify another lane's owned files or self-approve |
|
||||
| Certifier | Final independent evidence/traceability/gate decision | Merge, close provider issue, release, waive policy |
|
||||
|
||||
## 6. Gate model
|
||||
|
||||
### Mandatory gates
|
||||
|
||||
1. Requirements/contract freeze before parallel implementation.
|
||||
2. P0 schema/authority threat model and tenant isolation review.
|
||||
3. Author and reviewer MUST be different principals/sessions.
|
||||
4. Functional review validates requirements, endpoint registry, concurrency, and negative paths.
|
||||
5. **Mandatory SecReview (`secrev`)** for any auth, authorization, tenant, service-token, secret, database schema/migration, data-integrity, import/cutover, audit, lease/fencing, recovery, or destructive-retirement surface.
|
||||
6. Review findings enter bounded remediation owned by the implementation lane.
|
||||
7. Raising reviewer re-verifies remediation.
|
||||
8. Certifier performs the final independent evidence and traceability gate.
|
||||
9. Merge authority remains with `mos-claude`/Project Sub-Orchestrator control plane after gates pass.
|
||||
10. Post-merge CI and situational validation must be terminal green before closure.
|
||||
|
||||
### Gate outcomes
|
||||
|
||||
- **PASS:** evidence complete; next authority may proceed.
|
||||
- **REJECT:** findings are explicit and route to remediation.
|
||||
- **ESCALATE:** policy/owner decision required; no implicit waiver.
|
||||
|
||||
No role can transform a missing gate into a warning by changing status, editing a projection, or writing Valkey.
|
||||
|
||||
## 7. Slice ownership rules
|
||||
|
||||
1. USC web1 is the sole execution environment; coder2/3/4/5 are independent bounded lanes under Mos.
|
||||
2. Every slice has one named file-tree owner and an explicit IN/OUT boundary in `TASKS.md`.
|
||||
3. Two active slices MUST NOT edit the same source file, migration file, generated snapshot, lockfile, or API contract.
|
||||
4. coder2 exclusively owns `packages/db/src/schema.ts`, `packages/db/drizzle/**`, migration journal/meta/tests, then its disjoint recovery-parser/runbook slice. All schema requests serialize through coder2.
|
||||
5. Frozen `contracts/*.v1.ts` are read-only inputs during implementation. Contract changes require Mos approval, a version bump/amendment, and coordinated rebase before work resumes.
|
||||
6. coder3 exclusively owns Gateway DTO/controllers/services and the enumerated `apps/gateway/src/mcp/**` server files. coder4 owns CLI/projection clients and never edits MCP server files. Web consumers use the exact KBN-105 endpoint/DTO freeze.
|
||||
7. coder4 executes one lane order: CLI/projection → pure Coordinator → importer → cutover. The pure Coordinator under `packages/coord` does not load IDs or access DB, Gateway, Valkey, recovery I/O, or web files; coder3 owns the persistence/service adapter.
|
||||
8. Migration/import tooling calls Gateway/migration-only approved ports and does not add a second database model.
|
||||
9. Each lane commits only its owned files and reports any needed cross-slice change as a contract-change request instead of editing another lane's tree.
|
||||
10. Cross-review is mandatory: no lane reviews its own changes. Recommended ring is coder2 ← coder5, coder3 ← coder2, coder4 ← coder3, coder5 ← coder4, followed by independent SecReview where triggered and Certifier final.
|
||||
11. Integration-only edits are a separate serialized slice after component lanes are green; no opportunistic merge-conflict resolution may alter semantics.
|
||||
|
||||
## 8. Delivery phases and exit gates
|
||||
|
||||
### P0 — Canon and authority foundation
|
||||
|
||||
- Publish this canon, frozen schema/ports/health/recovery contracts, threat model, authorization matrix, exact endpoint/DTO registry, concrete current-main field-by-field migration map, and standards amendment.
|
||||
- Build hold remains active until independent author≠reviewer re-review returns GO on health proof/failures, approval binding, fencing, tenant relationships, proposals, migration map, slice ordering/API freeze, recovery validation, and vocabulary alignment.
|
||||
- Exit: no unresolved second writer or contract blocker, tenant boundary frozen, all seven decisions traceable, and independent re-review GO recorded.
|
||||
|
||||
### P1 — Thin native MVP
|
||||
|
||||
- Schema/migration, tenant-safe Gateway, CLI/MCP/projection, writable Kanban/List/Projects, dependencies/readiness/audit.
|
||||
- Exit: same revision across web/CLI/MCP/projection; cross-workspace tests fail closed; generated files cannot mutate state.
|
||||
|
||||
### P2 — Mechanical coordination
|
||||
|
||||
- Agent/session registry, deterministic engine, approval queue, PostgreSQL leases/fencing/checkpoints/outbox, retry/quarantine, operations UI.
|
||||
- Exit: one lease winner, stale tokens rejected, dependencies/approvals enforced, DB/Valkey fault semantics proven, Certifier gate has no merge authority.
|
||||
|
||||
### P3 — Shadow migration and cutover
|
||||
|
||||
- Importer, lineage, reconciliation, reviewer UI, write freeze, final delta, Gateway switch, legacy read-only, stabilization and rollback package.
|
||||
- Exit: signed reconciliation, zero active legacy writers, scoped Gateway identities, imported backlog cannot dispatch accidentally.
|
||||
|
||||
## 9. Evidence required for mission closure
|
||||
|
||||
- Requirement-to-test/evidence matrix.
|
||||
- Schema/migration and N-1 rolling-deploy proof.
|
||||
- Cross-workspace API/repository/import/Coordinator negative tests.
|
||||
- Health-state and fail-closed fault injection.
|
||||
- Valkey-loss/outbox replay and Coordinator restart tests.
|
||||
- Concurrent lease and stale fencing tests.
|
||||
- Endpoint-registry alignment across web/CLI/MCP/Gateway.
|
||||
- Accessible real-Gateway Kanban journeys.
|
||||
- Generated projection tamper/no-import proof.
|
||||
- One-way migration dry-run/apply/verify and field reconciliation.
|
||||
- Author-independent functional review and required SecReview.
|
||||
- Certifier final decision and evidence bundle.
|
||||
- Merged main SHA, terminal green CI, closed linked task/issue, and post-merge situational validation under orchestrator ownership.
|
||||
|
||||
## 10. Change control
|
||||
|
||||
This manifest is derived from the ratified source plan. Any change to SOT authority, workspace tenancy, fixed statuses, Coordinator/Certifier authority, health-state semantics, schema v1, migration direction, or recovery-tier field set is a contract change. Contract changes require Jason/Mos authorization and cannot be inferred by an implementation lane.
|
||||
|
||||
> KBN-101 Envelope A (rc.20, `KBN-101-DB-ROLE-SPLIT.md` §4/§10) is the specific authorized instance of a schema-v1 contract change under this clause, ruled by Jason B1 + Mos OPTION A/Q1/Q2; see `KBN-101-DB-ROLE-SPLIT.md` rc.20.
|
||||
|
||||
No coder lane may start while the build hold is active. KBN-010 must complete before KBN-100; KBN-105 exact endpoint/DTO freeze must complete before any API consumer implementation.
|
||||
@@ -0,0 +1,343 @@
|
||||
# Native Kanban/SOT — Remediated Shared Contract v1
|
||||
|
||||
**Status:** CONTROL-PLANE rc.16 KBN-101 current generic storage-wrapper authority remediation complete; awaiting independent exact-head re-review. Prior KCR-001–016 and rc.4 SI-001 decisions retained; KBN-101 foundation certification precedes KBN-100 and real immutable-operation certification precedes KBN-105
|
||||
**Version:** 1.0.0-rc.16
|
||||
**Date:** 2026-07-15
|
||||
**Change authority:** Mosaic control plane/Jason only
|
||||
**SI-001 amendment authority:** `web1:mosaic-100` control-plane decision under issue #753
|
||||
|
||||
## Amendment record
|
||||
|
||||
### 1.0.0-rc.20 — KBN-101 Envelope A: declarative sink-RBAC + per-role connection-selection + RLS write-source + sink-resident User-override
|
||||
|
||||
- **Choice:** adds the fixed User/Orchestrator/others runtime rung-roles (per-ROLE, deny-by-default; `mosaic_runtime_user`, `mosaic_runtime_orchestrator` + their capability roles), amends the frozen §4 L176 CONNECT allowlist for the two new logins (TEMPORARY-denied), and authorizes declarative RLS `WITH CHECK`/`USING` + `FORCE ROW LEVEL SECURITY` + a sink-resident `task_status_write_override` table on `tasks` — a schema-v1 mechanism addition beyond the prior grant/revoke-only model.
|
||||
- **No per-federated-user topology:** DB-role/credential cardinality remains fixed at 3, independent of federated-user count; the override table is keyed `(workspace_id, task_id, tier)`, never per-writer; per-writer attribution is a metadata column (`task_events.actor_id`), never a DB principal.
|
||||
- **Non-effect:** rc.4 SI-001, all KCR-001–016 decisions, and every prior rc (rc.5–rc.16) invariant not explicitly named above remain unchanged. Introduces no new principal, key, or per-user authority axis (Q1 HALT boundary not hit).
|
||||
- **Authority:** Jason B1 ruling + Mos OPTION A (home the layer) + Mos Q1 (TIER-LEVEL) + Mos Q2 (RLS `WITH CHECK` authorized, two mandatory conditions — `ENABLE`/`FORCE ROW LEVEL SECURITY` as terminal migration step, non-owner rung roles). Exact implementation detail is normative in [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md) §4/§10 and the KBN-101 Envelope A v6 record.
|
||||
- **Gate:** lands as `KBN-101-10` (new owner card) + responsibility-widenings on `KBN-101-00/01/02/03/05/06`; the `tasks`/override-table DDL itself is homed in the **KBN-100** producer (starts after KBN-101-08) per the existing §4 L178 producer/consumer precedent — a SPEC note today, not an immediate implementation.
|
||||
|
||||
### 1.0.0-rc.16 — Current generic storage-wrapper authority closure
|
||||
|
||||
- **Current-source truth:** `packages/storage/src/cli.ts` currently shells `storage migrate --run` directly to `pnpm --filter @mosaicstack/db db:migrate` through `execSync`; no `mosaic-db-migrator` executable exists. README and user-guide command guidance therefore remove that command and any runner-delegation claim. The current wrapper is legacy N-1, uncertified, non-operative, and MUST NOT be invoked pending KBN-101-02/-03/-06/-08 activation.
|
||||
- **Future-only boundary:** future schema migration remains non-operative and follows external bootstrap → TLS/roles → runner `--run` → runner `--verify` → readiness; tier copy uses only the separately held secure migrate-tier route.
|
||||
- **Unmaskable semantic/source-consistency evidence:** before inventory, ownership, or status masking, -06 fails the exact former README commented code-fence generic-wrapper form and exact user-guide executable generic-wrapper form. Its source-consistency test proves the direct-Drizzle `execSync` target and absent runner bin, so any documentation describing current wrapper delegation to the runner fails.
|
||||
- **Non-effect:** prior runner, legacy-CI, Compose, production-secret, attestation, pgvector, manifest, lock, TLS, activation, and serial-gate closures remain unchanged.
|
||||
|
||||
### 1.0.0-rc.15 — Held runner and legacy-CI authority closure
|
||||
|
||||
- **Held runner only:** Current operator documents cannot advertise `mosaic-db-migrator --run|--verify` as executable. The sole passing future form is one `Held future procedure` Markdown section, bounded through its next equal-or-higher heading, that explicitly says non-operative/no-current-command-authority, names KBN-101-00/-03/-05, and preserves external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway/Compose readiness. Any runner hit outside that section fails before inventory/ownership/status masking.
|
||||
- **PGlite/current-CI boundary:** Fleet backlog current behavior is PGlite-only; PostgreSQL CLI/runner authority remains held until activation. README classifies the checked-in direct `db:migrate` CI job as active legacy N-1, uncertified, non-authorizing as an operator route, and pending KBN-101-06 removal; it is a known direct-DDL exception against an isolated disposable CI database, not approved ordinary behavior. The -06 fixture asserts every required status term and rejects ordinary-authority presentation.
|
||||
- **Non-effect:** prior Compose, production-secret, attestation, pgvector, manifest, lock, TLS, activation, and serial-gate closures remain unchanged.
|
||||
|
||||
### 1.0.0-rc.14 — Current Compose and production-secret route closure
|
||||
|
||||
- **Current developer boundary:** `README.md` and `docs/guides/dev-guide.md` permit only in-process PGlite data-layer work and explicitly selected non-PostgreSQL Compose services. Gateway/Web local start is held because the current unguarded loader can inherit a daemon/project PostgreSQL DSN and reach runtime DDL; KBN-101-02 must reject it before connection. The current PostgreSQL Compose mount is legacy/unqualified; PostgreSQL and federated activation are held until KBN-101-00/-03/-05 and then follow external bootstrap → TLS/roles → runner `--run` → `--verify` → Gateway/Compose readiness.
|
||||
- **Production boundary:** `docs/guides/deployment.md` is non-operative until the KBN-101-05 renderer-backed process-exec or `LoadCredential` interface exists. It contains no active production environment-file, monorepo auto-load, credential export/argv, or secret-activation lifecycle route; future units must preserve generation-pinned Vault consumer isolation.
|
||||
- **Unmaskable semantic negatives:** -06 fails the exact former README/dev/deployment Compose-first sequences and every production `.env`, `EnvironmentFile=`, credential export/argv, or restart-as-secret-activation fixture before owned/status/normative classification. The held PGlite/non-PostgreSQL route and future ordered activation are the only passing fixtures.
|
||||
|
||||
### 1.0.0-rc.13 — Federation-MILESTONES indirect-startup closure
|
||||
|
||||
- **Complete operator inventory:** `docs/federation/MILESTONES.md` is exclusively KBN-101-07 and an exact KBN-101-06 `operator-document` `status-only` record. Its former `pgvector extension installed + verified on startup` wording is superseded and forbidden; it authorizes no current DDL, Compose/init, or runtime/startup path.
|
||||
- **Unmaskable semantic negative:** before inventory disposition, the scanner fixture proves that exact former wording fails. The only passing status-only sequence is external bootstrap → TLS/roles → `mosaic-db-migrator --run` → `mosaic-db-migrator --verify` → Gateway readiness.
|
||||
|
||||
### 1.0.0-rc.12 — Deployable importer generation and indirect-DDL-route closure
|
||||
|
||||
- **Authenticated generation:** KBN-101-05 owns one canonical Vault KV-v2 importer record, `secret-{env}/mosaic-stack/database/importer` key `url`, with its version taken only from the same successful `data.metadata.version` response. Value plus provider version are one generation, never inferred from DSN bytes. The renderer creates separate immutable `0400` URL/version copies for migrator `10003:10003` binding-only access and importer `10002:10002` access; it uses fsync/atomic generation replacement for Compose and distinct versioned secret/config references for Swarm, so deployment cannot mix generations.
|
||||
- **Bounded consumers:** importer alone receives its URL/version, CA at `DATABASE_TLS_CA_CERT_PATH`, pinned public key, and read-only attestation; migrator receives its own migration URL/CA, the URL/version only for no-connect/no-export binding, attestation output, and the root-wrapper-only private key. Safe fd open/fstat/digest/zeroize/close semantics, a privileged producer-only-to-importer-only attestation handoff controller (verify, exact-byte copy, fsync/atomic rename, `10002:10002` `0400` seal, then importer start), no shared writable file, no logging/oracle, provider rotation/revocation, CA/mount, consumer-isolation, and symlink/hardlink/owner/mode/TOCTOU negatives are mandatory.
|
||||
- **Indirect-DDL closure:** `docs/federation/SETUP.md` is non-operative until KBN-101 activation and documents only external bootstrap → TLS/roles → runner `--run` → `--verify` → Gateway readiness. The -06 scanner performs unsuppressible semantic checks for automatic first-boot/startup extension/schema/migration language, Compose-up-before-runner, and init-script authority; the former SETUP wording fails and the remediated sequence passes.
|
||||
|
||||
### 1.0.0-rc.11 — Target-bound importer attestation and exhaustive operator-route closure
|
||||
|
||||
- **Target-bound proof:** trusted `mosaic-db-migrator --verify` now produces the atomic, credential-free `migrate-target.v1.json` JCS/Ed25519 artifact from a runner-only root-owned signing-key reference; the importer receives only pinned public verification keys and the artifact. Its signed v1 fields bind issued/expiry/nonce, exact secret version and SHA-256 of high-entropy target-file bytes, canonical TLS host/port/database, CA/SPKI, PostgreSQL system identifier/database OID, expected importer role, manifest/schema fingerprints, and producer invocation/build/image/correlation. No DSN, username, password, credential bytes, or signing key enters the artifact, importer, runtime, logs, or output.
|
||||
- **Fail-closed importer:** `mosaic storage migrate-tier` requires both `--target-url-file /run/secrets/mosaic-migrate-target-url` and `--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json`. Before target connection it validates files, signature/key/expiry/replay, secret version/digest, TLS/CA/role/manifest bindings and opens/digests/connects from the same in-memory URL bytes. After verified TLS but before transaction/DML it matches server ID, database OID, `current_user`, CA/SPKI, and manifest/schema; failure distinguishes zero connection from connection/zero-DML and DDL remains impossible. Rotation overlap/revocation, atomic rename, replay cache, secret rotation invalidation, and wrong/substituted/stale/tampered/file-change tests are mandatory.
|
||||
- **Closed documentation surface:** KBN-101-06 inventories every current non-normative scanner hit, including `docs/guides/user-guide.md` and status-only `docs/federation/TASKS.md`; the latter is historical and cannot authorize DDL. The legacy `storage migrate` tier-copy syntax is unavailable. `storage migrate` is schema-wrapper delegation only; secure tier data copy is `migrate-tier`. Exact KBN PRD/contract/shared/task paths may be `normative-contract` scan class but are still scanned and cannot mask executable instructions. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
- **Non-effect:** pgvector closure, manifest, lock, role graph, TLS, activation, and KBN-100/KBN-105 serial gates are unchanged.
|
||||
|
||||
### 1.0.0-rc.10 — PostgreSQL-valid untrusted pgvector owner and active migrate-tier closure
|
||||
|
||||
- **Valid extension authority:** PostgreSQL 17 + pgvector 0.8.2 `vector` is untrusted (`trusted` absent; `relocatable=true`), so `mosaic_extension_owner` is exactly `NOLOGIN SUPERUSER`, not `NOSUPERUSER`. It is dedicated solely to `mosaic_extensions`, `vector`, and owner-bearing extension members; `rolcanlogin=false`, `rolsuper=true`, zero members, no runtime credential/Vault secret, and no app-container delivery are catalog and deployment proof. An externally controlled audited bootstrap-superuser session alone `SET ROLE`s for extension CREATE/UPDATE/SET SCHEMA, then `RESET ROLE`; fresh and shadow paths do so, while in-place existing work requires exact pre-existing `extowner`.
|
||||
- **Explicit superuser exception:** `GRANT`/`REVOKE` cannot privilege-limit a superuser. The containment is dedicated identity, no login, no membership, external control plane, audit, independent review, backup/rollback, and maintenance window—not a false least-privilege claim. Runtime, migrator, schema owner, importer, and every service role cannot assume the role or alter/update/drop/change extension membership. Managed targets without this exact role are ineligible unless a versioned provider-owned extension-owner profile is independently approved.
|
||||
- **Active secure data-migration route:** `docs/guides/migrate-tier.md` is exclusively KBN-101-07, is active rather than historical, and specifies runner-prepared/verified PostgreSQL destination plus a dedicated non-DDL importer. KBN-101-02 freezes `--target-url-file /run/secrets/mosaic-migrate-target-url`, never credential argv; raw `--target-url`, `DATABASE_URL` fallback, runtime owner, missing/unsafe file, wrong mode, and DDL all fail before target connection/DDL. KBN-101-06 inventory/matrix records the route and exact secure fields, then tests its finite operator-document closure.
|
||||
- **Non-effect:** manifest, lock, `mosaic` application schema, TLS, activation, and KBN-100/KBN-105 serial gates remain unchanged. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
|
||||
### 1.0.0-rc.9 — KBN-101 extension-schema boundary, disjoint manifests, and scanner mechanics
|
||||
|
||||
- **Extension schema owner:** `mosaic_extension_owner`, not `mosaic_schema_owner`, creates and owns `mosaic_extensions`, `vector`, and extension-member objects. The external bootstrap actor `SET ROLE`s for fresh creation or approved-owner relocation, then `RESET ROLE`s; rc.10 replaces the earlier membership wording with the PostgreSQL-valid zero-member superuser exception. Schema owner has only `USAGE` for legacy type resolution—never ownership, `CREATE`, `ALTER`, `DROP`, member change, or default-privilege authority. Runtime, migrator, and schema owner must fail catalog and direct DDL denials; shadow/resume/rollback repeat the owner/default-privilege proof.
|
||||
- **Exclusive delivery DAG:** KBN-101-00…09 now has a complete, nonoverlapping exact file/glob manifest with named tests/evidence. The runner mapping is exactly `"mosaic-db-migrator": "./dist/cli.js"` and image `ENTRYPOINT ["mosaic-db-migrator"]`; `packages/storage/src/{cli,migrate-tier}.ts` belongs only to -02, and -07 is documentation only. -08/-09 own evidence paths only. -00…07 are prepared artifacts; the immutable N-1 image remains live until -08 atomic activation, so no independently deployed intermediate can bypass runtime controls.
|
||||
- **Mechanical classifier:** -06 owns the exact scanner, inventory fixture, command-matrix harness, and CI wiring. Inventory records pin path/class/owner/disposition/allowed tokens/rationale/expiry/review revision; unknown, duplicate-owner, ownerless, missing-path, invalid allowlist, and historical-category masking fail. The architecture plan's operative direct `db:migrate` is replaced by sole-runner guidance rather than hidden under a historical category.
|
||||
- **Non-effect:** manifest v1, lock, `mosaic` application-schema ownership, TLS, activation, KBN-100/KBN-105 serial gates, and all earlier canon decisions remain unchanged. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
|
||||
### 1.0.0-rc.8 — KBN-101 finite authority, executable runner, and pgvector-owner remediation
|
||||
|
||||
- **Finite authority closure:** KBN-101-06 classifies every current executable source/script/package bin, operator document, and deploy manifest by exact path; unclassified current hits fail. Byte-immutable historical SQL, PGlite-only routines, negative-test literals, vendored/generated artifacts, and clearly labeled historical reports are exact-path/category reviewed allowlists only. `packages/db/src/index.ts` loses its public `runMigrations` export with a direct-import/compile negative; `docs/fleet/backlog-conventions.md` and `docs/PERFORMANCE.md` lose first-use/direct-Drizzle/Gateway-startup migration instructions and carry runner/readiness route negatives. A token scan is only input to the classifier, never proof of authority.
|
||||
- **Executable exclusive cards:** KBN-101-03 alone publishes `mosaic-db-migrator` from `packages/db/package.json`/`src/cli.ts`, owns `docker/db-migrator.Dockerfile`, and keeps `{runner,config.dto,manifest,identity,tls}` private, with exact `--run|--verify|--help`, env-only input, stable exits, and command tests. KBN-101-00 alone owns `infra/pg-bootstrap/roles.sql`, `infra/pg-bootstrap/extensions.sql`, `infra/pg-bootstrap/README.md`, plus bootstrap tests. KBN-101-05 alone owns `tools/db/render-postgres-secrets.ts`, renderer tests, and Compose/Portainer/Swarm/two-gateway declarations, consuming the versioned bootstrap interface. No card overlaps renderer/bootstrap/deployment ownership.
|
||||
- **Extension-owner transition:** `mosaic_extension_owner` is a dedicated NOLOGIN role whose membership/credentials never reach services; the external bootstrap actor alone may `SET ROLE` during bootstrap. Fresh vector and member objects retain that owner. PostgreSQL has no supported extension-owner alteration: approved-owner existing extension relocation validates `pg_extension.extowner`, members/schema/version and uses tested `ALTER EXTENSION ... SET SCHEMA`; legacy runtime-owned extension fails closed to a controlled shadow database migration with backup, evidence, quiesce/final delta, atomic switch, and read-only rollback window. No catalog mutation, ownership adoption, or `DROP CASCADE` is permitted. Runtime/migrator/schema-owner extension ALTER/DROP/member-update denial is mandatory.
|
||||
- **Non-effect:** manifest v1, lock namespace, role/search-path, relocation/TLS/activation, KBN-100/KBN-105 serial gates, and all retained canon decisions are strengthened, not weakened. The normative detail remains [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
|
||||
### 1.0.0-rc.7 — KBN-101 complete current-path, relocation, and two-gateway remediation
|
||||
|
||||
- **Finite current-path closure:** static inventory and the `DATABASE_URL`-only-before-connect/DDL denial matrix now explicitly include Gateway's former temporary-table pgvector test (runner-prepared persistent read/query-only fixture), `docker/init-db.sql` retirement, `migrate-tier.ts` runner/bootstrap-only guidance, and the active two-gateway harness. The harness is migrated, not retired: `postgres-a/b → mosaic-db-migrator-a/b → gateway-a/b`, each with isolated URL/CA material, verified readiness, SANs, and positive/negative TLS evidence.
|
||||
- **Executable relocation:** KBN-101-03 exclusively owns `schema.ts`, Drizzle snapshots/journal/generated relocation and exact tests. All future application declarations use exported `pgSchema('mosaic')`; immutable historical SQL runs only in trusted legacy `public`. `vector` is fixed in non-writable `mosaic_extensions`, with exact catalog relocatability/version eligibility, explicit type/operator qualification, catalog-class ordering, unknown-object fail-closed behavior, clean/current-public/partial/reverse rollback tests, and an N-1 release order.
|
||||
- **Bound deployment ownership:** `mosaicstack/stack` KBN-101-00/05 owns current Compose, Portainer, two-gateway, bootstrap renderer/templates, UID/GID declarations, and rendered validation. Gateway is fixed to `10001:10001`; PostgreSQL UID/GID is image-inspected and frozen only after digest pinning. Exact secret paths, atomic renderer behavior, Compose/Swarm targets/modes, Gateway/PostgreSQL leaf separation, and two-pair TLS failure evidence are required. Mosaic deployment control plane/Jason is the named activation authority; environment IaC/Vault supplies versioned input only.
|
||||
- **Correct traceability:** REQ-03 maps to role/schema/search-path, REQ-04 to TLS, REQ-05 to post-KBN-100 immutability, REQ-06 to rollout/rollback, and REQ-07 to the KBN-101 → KBN-100 → KBN-101 → KBN-105 sequence. No prior manifest/lock/role/DAG/activation decision is weakened.
|
||||
|
||||
### 1.0.0-rc.6 — KBN-101 closed DDL/TLS/ledger activation remediation
|
||||
|
||||
- **Choice:** `mosaic-db-migrator` is the sole application/CI/test PostgreSQL DDL control plane. Every legacy entrypoint is routed or denied, rejects `DATABASE_URL`-only before connection/DDL, and `db:push` is unavailable outside an allowlisted disposable developer target. The runner holds one `max:1` session with fixed `pg_try_advisory_lock(1297044289,1262636593)` across preflight through release.
|
||||
- **Exact ledger:** manifest v1 canonically serializes journal logical index/tag and SHA-256 of exact shipped migration bytes. It maps each observed ledger hash to one tuple; physical insertion order is non-normative, while missing/unknown/duplicate/ambiguous/corrupt/stale states fail closed. Shipped `0009` bytes remain unchanged; a missing/effects-absent `0009` runs normally, an applied-late hash maps normally, and partial/full effects with missing hash require backup restoration or separately reviewed repair—not manual adoption.
|
||||
- **TLS/search path:** operator/IaC owns CA and server leaf lifecycle, exact compose/Swarm secret mounts, server TLS activation, service-DNS SANs, verified-TLS readiness, transition, CA overlap rotation, and rollback. Runtime/migrator use `verify-full`; PGlite is not PostgreSQL TLS evidence. Application sessions use only `pg_catalog,mosaic`; no URL/config-derived identifier reaches SQL.
|
||||
- **Safe release:** cards 00–07 land prepared but inactive; owner-runtime deployments remain N-1. Mosaic control plane/Jason alone authorizes one atomic TLS/roles → runner → readiness → runtime activation or rollback. No runtime-operator compatibility switch, bypass, plaintext interval, or force-on-red exists; all temporary support is removed before KBN-101-08.
|
||||
- **Non-effect:** role graph (**except as amended by rc.20 — see below**), immutable certification after KBN-100, KBN-105 gate, rc.5’s preserved rc.4 SI-001 invariants, and all KCR-001–016 decisions remain unchanged. Exact detail is normative in [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
|
||||
### 1.0.0-rc.5 — KBN-101 role/connection split
|
||||
|
||||
- **Choice:** PostgreSQL `standalone` and `federated` runtime uses `DATABASE_URL` only as a non-owner `mosaic_runtime` login; an explicit migration phase uses `DATABASE_MIGRATION_URL` only as `mosaic_migrator`, which `SET ROLE`s to non-login `mosaic_schema_owner` for DDL. Local PGlite remains an explicit embedded exception.
|
||||
- **No fallback / no startup DDL:** missing migration URL fails the migration phase; it never falls back to runtime URL/default/config. Gateway replicas do not run migrations. An advisory-locked migration phase verifies the exact ordered Drizzle ledger fingerprint before replicas may become ready.
|
||||
- **Privilege model:** non-login `mosaic_platform_database_owner` is outside application paths; `mosaic_schema_owner` owns only application/ledger schemas. `mosaic_runtime` has only `mosaic_runtime_capability`, owns no object/schema, cannot assume owner/migrator, has no TEMPORARY privilege, has only read access to the Drizzle ledger, and must fail startup if effective identity, unsafe attributes, authenticated TLS, search path, schema version, grants, or immutable relation privileges differ from the frozen contract. `task_events`, `artifacts`, `task_checkpoints`, `task_checkpoint_artifacts`, and `approval_decision_artifacts` grant runtime only INSERT/SELECT; KBN-100 retains RESTRICT/no-cascade semantics.
|
||||
- **Non-effect:** rc.4 SI-001 candidate-key/FK order and all KCR-001–016 tenancy, SOT, proposal-audit, approval, fence, recovery, no-cascade, endpoint, and wire invariants are unchanged. This amendment neither creates roles/secrets nor changes production deployment (**except as amended by rc.20 — see below**).
|
||||
- **Gate:** KBN-101’s role/schema-boundary foundation certificate, Vault/redaction/rotation, N-1/rollback, and independent security GO are mandatory before KBN-100. After KBN-100 creates the immutable relations, KBN-101 real deployed-role immutable-operation certification plus Ultron GO is mandatory before KBN-105; synthetic test-role success alone is insufficient. Exact implementation detail is normative in [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md).
|
||||
|
||||
### 1.0.0-rc.4 — KBN010-SI-001 (preserved)
|
||||
|
||||
- **Choice:** add the explicitly named, non-partial unique candidate key `missions_workspace_id_uidx` on `missions(workspace_id, id)` and retain `missions_workspace_project_id_uidx` on `(workspace_id, project_id, id)`.
|
||||
- **Rationale:** mission `id` remains globally unique, while the composite candidate key makes the frozen tenant-safe generic mission relations valid. `artifacts` and `approval_decisions` are polymorphic exactly-one-target records and do not consistently carry `project_id`; widening both children would unnecessarily broaden v1 and its target semantics.
|
||||
- **Exact effect:** `artifacts_workspace_mission_fk` and `approval_decisions_workspace_mission_fk` continue to reference the exact ordered columns `missions(workspace_id, id)` with RESTRICT deletion, now backed by a matching candidate key.
|
||||
- **Non-effect:** no SOT, tenancy, project-congruence, proposal-audit, approval, fencing, immutability, no-cascade, API, or wire-version invariant changes. The `SuccessEnvelopeV1.contractVersion` remains `1.0.0`.
|
||||
- **Historical evidence boundary:** `KBN-010-THREAT-AUTH-CONSTRAINT-GATE.md` intentionally remains the immutable rc.3 blocker verdict that detected SI-001; this rc.4 record and the #753 scratchpad append are the authorized disposition. Rewriting the gate verdict is outside this amendment's exclusive scope.
|
||||
- **Gate:** this amendment resolves the DDL defect identified by KBN010-SI-001 but does not itself lift KBN-100; independent schema/SecReview remains required.
|
||||
|
||||
## 1. Authority
|
||||
|
||||
Concrete contracts are the four `contracts/*.v1.ts` files. PostgreSQL/current-main Drizzle is the sole writable SOT. In PostgreSQL standalone/federated deployments, KBN-101 rc.13 DDL/ledger/TLS/role/attestation/generation separation is a precondition to schema implementation and certification. Public health, Valkey, files, exports, providers, browser state, and outage notes cannot authorize/reconstruct writes. Mechanical Coordinator is non-LLM with no scope/gate/certification/merge authority. Certifier is final independent gate with no merge authority. No feature lane starts until this canon merges and the KBN-010/KBN-105 prerequisites are satisfied.
|
||||
|
||||
## 2. Health proof and exact failures
|
||||
|
||||
`KanbanHealthResponseV1` is a discriminated union:
|
||||
|
||||
| State | read | write | Capability |
|
||||
| -------------------- | ----: | ----: | --------------------------------------------------- |
|
||||
| `healthy` | true | true | reads; public state still cannot authorize mutation |
|
||||
| `read-only-degraded` | true | false | reads only |
|
||||
| `write-unavailable` | false | false | diagnostics only |
|
||||
|
||||
Every response has `checkedAt`, `validUntil`, `policyRevision`; contradictory booleans fail validation.
|
||||
|
||||
For a mutation, Gateway opens the PostgreSQL transaction, executes the live write probe on that transaction/connection, mints the internal branded `PostgresWriteHealthProofV1`, and revalidates time/policy/transaction identity immediately before mutation. Public REST/MCP/CLI DTOs never accept health/proof fields. Valkey/caller assertions cannot mint proof. Pure Coordinator takes `KanbanEvaluationContextV1`; persistence takes `InternalKanbanMutationContextV1` or probes internally.
|
||||
|
||||
| Case | HTTP | Frozen result | Retry |
|
||||
| ---------------------------- | --------------------------: | ------------------------------------------------------------------- | -------------------- |
|
||||
| degraded write | 503 | `KANBAN_WRITE_HEALTH_UNPROVEN`, `read-only-degraded`, `not_applied` | false |
|
||||
| write unavailable | 503 | `KANBAN_WRITE_UNAVAILABLE`, `write-unavailable`, `not_applied` | false |
|
||||
| version conflict | 409 | `AGGREGATE_VERSION_CONFLICT`, actual version, `not_applied` | false |
|
||||
| timeout/unreachable | timeout/502/504 | `retryable_transport_error`, `unknown` | same idempotency key |
|
||||
| stale fence/session/approval | coordinator rejection union | `not_applied` | false |
|
||||
|
||||
Required negatives: contradictory state, expired/policy-mismatched/wrong-transaction proof, Valkey-only health, forged healthy, and exhaustive non-cross-mapping of 503 vs 502/504/timeout vs 409.
|
||||
|
||||
## 3. Canonical schema invariants
|
||||
|
||||
Complete declaration: `contracts/kanban-schema.v1.ts`.
|
||||
|
||||
- Tables: tenant/identity (`workspaces`, members, teams/members, agents/sessions); planning (`projects`, `milestones`, current-milestone join, `missions`, mission-milestones, `tasks`, normalized tags, dependencies); orchestration (`task_assignments`, durable execution state, leases, checkpoints/evidence); governance (`change_proposals`, immutable artifacts/evidence, events, approvals, outbox, external links).
|
||||
- Task statuses: `backlog | ready | in_progress | blocked | in_review | done | cancelled`.
|
||||
- Assignment states everywhere: `awaiting_approval | policy_pre_authorized | approved | rejected | leased | released | expired | superseded`.
|
||||
- Specialist roles everywhere: `planning | enhance | coder | review | security-review | pr-monitor | certifier`.
|
||||
- Owner uses exactly-one user/team; assignment principal exactly-one user/team/agent; users require active membership; agent/session and all evidence are workspace-bound.
|
||||
- Task→mission/milestone/parent, mission→milestone, and project→current-milestone are project-congruent composite relations.
|
||||
- Mission `id` remains globally unique. The additional non-partial `missions_workspace_id_uidx` candidate key on `(workspace_id, id)` exists only to support the frozen workspace-safe polymorphic artifact and approval-decision mission relations; the project-congruent `(workspace_id, project_id, id)` key remains authoritative wherever `project_id` is present.
|
||||
- Dependency identity is workspace+predecessor+successor independent of type.
|
||||
- Approval evidence and checkpoint evidence are workspace-scoped joins to immutable artifacts, never JSON ID arrays.
|
||||
- Proposal audit links are composite relations: `(workspace_id, submitted_audit_event_id)` and `(workspace_id, accepted_command_audit_event_id)` reference `task_events(workspace_id, id)` with RESTRICT deletion.
|
||||
- Assignment is persisted with task/version, exact target/session, expiry/state/policy/proposer/reason. Approval relates to assignment. Lease acquisition accepts IDs, then reloads/locks and validates every relation.
|
||||
- `tasks.fencing_counter` is bigint; locked atomic increment/RETURNING creates a decimal-string lease token. Lease/checkpoint composites bind exact workspace+task+assignment/session+fence.
|
||||
- `task_execution_states` durably records retry/quarantine/exhaustion.
|
||||
- Tags are normalized; legacy `tasks.tags` remains through N-1. Archive is explicit actor/reason/time and does not change lifecycle.
|
||||
- Canonical parents use RESTRICT. Events/checkpoints/artifacts/evidence are INSERT/SELECT-only for application roles. Normal flow archives/cancels; purge is audited break-glass retention work.
|
||||
|
||||
## 4. Outage proposal contract
|
||||
|
||||
`change_proposals` stores workspace, active-member proposer, source-note digest, target/version, typed command/payload, idempotency, lifecycle, decision actor/reason/time, proposal version, and submit/accepted event IDs. Both event IDs are workspace-aware composite foreign keys to `task_events(workspace_id, id)`; a bare UUID is never sufficient.
|
||||
|
||||
Submission preallocates the proposal ID. One transaction inserts `change_proposal.submitted` with the proposal workspace, `aggregate_type='change_proposal'`, `aggregate_id=<new proposal ID>`, `previous_version=NULL`, and `new_version=1`, then inserts the proposal referencing that event. Missing, foreign-workspace, wrong-type, or unrelated-proposal events abort the transaction.
|
||||
|
||||
Submit/list/get/accept/reject are explicit Gateway commands. Pending/rejected proposals are inert: no scheduling, dependency/gate satisfaction, or direct target mutation. Acceptance locks proposal+target, obtains fresh transaction-local proof, verifies pending/expected version, invokes the normal command handler, and atomically stores the emitted normal-command event ID. That event must share the proposal workspace, match `target_aggregate_type` and `target_aggregate_id`, use `causation_id=submitted_audit_event_id`, and carry `payload.changeProposalId=<locked proposal ID>`. Missing, foreign-workspace, unrelated-target, unrelated-proposal, or unrelated-command events abort acceptance.
|
||||
|
||||
## 5. Concrete current-main N-1 migration delta
|
||||
|
||||
**Inspected:** `origin/main:packages/db/src/schema.ts` at `e72388b2cbfe400842fe940fa6cabf984ed43711` (2026-07-13). It has global teams/no workspace keys, legacy project/mission/task statuses, nullable task project/mission, `tasks.assignee/tags/due_date`, mission JSON/config, duplicated `mission_tasks.status`, legacy agent fields, and separate fleet `backlog` claims.
|
||||
|
||||
Legacy columns remain declared in unified `schema.ts` for expand + full N-1/rollback window. Generation must not infer early drops.
|
||||
|
||||
### 5.1 Ordered phases
|
||||
|
||||
1. **Pre-expand:** N-1 patch stops `mission_tasks.status` as write source; inventory writers; backup/checksum.
|
||||
2. **Expand:** add enums/tables and nullable-first columns; retain legacy declarations/uniques; emit no v1-only status.
|
||||
3. **Backfill:** bootstrap workspace; bounded idempotent cursor/checksum batches; quarantine ambiguous rows.
|
||||
4. **Validate:** no null tenant, cross-project link, ambiguous owner; status/tag/date/config retention; then constraints/NOT NULL.
|
||||
5. **Compatibility:** N-1 reads legacy; same-DB transaction mirrors only unavoidable fields; never file/Valkey dual write.
|
||||
6. **Switch:** stop N-1 writers; Gateway sole command boundary; enable canonical statuses.
|
||||
7. **Contract release:** later release after rollback/N-1; remove compatibility/global uniques/legacy fields.
|
||||
|
||||
### 5.2 Mission candidate-key and dependent-FK DDL order
|
||||
|
||||
KBN-100 migration DDL must execute the SI-001 portion in this order:
|
||||
|
||||
1. expand/backfill `missions.workspace_id` and `missions.project_id` while preserving the global `missions.id` primary key and the project-congruent `missions_workspace_project_id_uidx` key;
|
||||
2. prove duplicate-key feasibility on the production-shape dataset: `(workspace_id, id)` has no duplicate groups and global `id` uniqueness remains intact;
|
||||
3. create the non-partial unique index `missions_workspace_id_uidx` on exact ordered columns `(workspace_id, id)`;
|
||||
4. only after step 3, create/alter `artifacts` and add `artifacts_workspace_mission_fk` from `(workspace_id, mission_id)` to exact `missions(workspace_id, id)` with `ON DELETE RESTRICT`;
|
||||
5. only after step 3, create/alter `approval_decisions` and add `approval_decisions_workspace_mission_fk` from `(workspace_id, mission_id)` to exact `missions(workspace_id, id)` with `ON DELETE RESTRICT`;
|
||||
6. validate both constraints and prove a mission ID paired with a foreign workspace is rejected for each child.
|
||||
|
||||
The candidate key is intentionally redundant with globally unique `missions.id`, but PostgreSQL requires a matching unique candidate key for the exact two-column FK target. It is additive and N-1-safe. Pre-switch rollback drops the two dependent FKs/tables before dropping this candidate key, preserves the global primary key and project-congruent key, and follows the existing freeze/reconciliation rule after the first canonical mutation.
|
||||
|
||||
### 5.3 New audit/proposal DDL order
|
||||
|
||||
KBN-100 migration DDL may begin only after KBN-101 foundation role/schema-boundary certification. It runs in the explicit migrator/owner phase—not Gateway startup—and its generated Drizzle declaration/snapshot/journal must be mutually consistent. It must execute in this order:
|
||||
|
||||
1. create `task_events` and its unique `(workspace_id, id)` key;
|
||||
2. create `change_proposals` with nullable acceptance-event ID and required submission-event ID;
|
||||
3. add `change_proposals_workspace_submitted_event_fk` from `(workspace_id, submitted_audit_event_id)` to `task_events(workspace_id, id)` with `ON DELETE RESTRICT`;
|
||||
4. add `change_proposals_workspace_accepted_command_event_fk` from `(workspace_id, accepted_command_audit_event_id)` to the same composite key with `ON DELETE RESTRICT`;
|
||||
5. install application-role immutability privileges and same-transaction semantic validation before enabling proposal commands.
|
||||
|
||||
The submission transaction inserts the event first using a preallocated proposal UUID, then the proposal. Acceptance inserts the normal command event before updating the locked proposal. Neither FK is omitted or replaced by a bare UUID/index check.
|
||||
|
||||
### 5.4 Field map
|
||||
|
||||
| Current | Expand/backfill | N-1 compatibility | Switch/contract |
|
||||
| ---------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | ---------------------------------------------------------------------------------- |
|
||||
| global `teams`, `team_members` | add workspace nullable; bootstrap; validate active owners | retain global slug/FKs | workspace composites; global unique contracts later |
|
||||
| `projects.status` | add `canonical_status`; map active/paused/completed/archived | mirror representable values; no `planning` | canonical authority; legacy contracts later |
|
||||
| project `owner_id/team_id/owner_type` | add exact accountable user/team; deterministic map or quarantine | preserve old reads and compare drift | canonical exact-one; remove legacy after parity |
|
||||
| current milestone | create milestones then join table (no circular DDL) | absent to N-1 | join is authority |
|
||||
| nullable `missions.project_id` | derive workspace/project; null/orphan exception, never guess | keep nullable legacy read | canonical required; validate/set NOT NULL later |
|
||||
| mission relational candidate keys | retain global `id` PK and project-congruent key; add non-partial `(workspace_id,id)` key before artifact/approval FKs | additive key is ignored safely by N-1 readers/writers | retain both composite keys; generic mission children use exact workspace+ID target |
|
||||
| mission `description` | add objective; preserve description; reviewed nonblank mapping | N-1 description | objective authority; retain until signed review |
|
||||
| `missions.status` | add canonical; planning→draft, active/paused/completed/failed same | no new-only statuses emitted | canonical authority |
|
||||
| mission `milestones` JSON | normalize with source digest; preserve malformed/original | N-1 reads JSON; no reverse sync | normalized authority; JSON removed after checksum sign-off |
|
||||
| mission config/metadata/phase/user | retain all; map known typed policy only | all remain declared | remove only by signed consumer inventory |
|
||||
| nullable `tasks.project_id` | derive explicit/mission project; orphan quarantine | retain nullable read/write during compatibility | canonical required; NOT NULL later |
|
||||
| `tasks.mission_id` | add project-congruent composite | old relation readable | composite authority |
|
||||
| `tasks.status` | canonical: not-started→backlog, in-progress→in_progress, others same | no ready/in_review emission | canonical authority |
|
||||
| `tasks.assignee` | deterministic active user/team/agent assignment; raw value preserved if ambiguous | mirror text only if unambiguous | canonical owner/assignment; remove after no-loss sign-off |
|
||||
| `tasks.tags` JSON | normalize trim/case/dedupe with original digest | transactionally mirror normalized rows | normalized authority; JSON later removed |
|
||||
| `tasks.due_date` | copy exactly to `due_at` | mirror | due_at authority; legacy later |
|
||||
| task common fields | preserve metadata byte-for-byte; add criteria/rank/retry/archive/version/fence | old reads valid | new fields canonical |
|
||||
| `mission_tasks.status` | keep; prohibit as write source; linked status ignored; unlinked becomes task or reject | read-only compatibility value | membership uses task mission; status dropped after no readers |
|
||||
| mission-task notes/PR/user | map to metadata/artifact/event/link/attribution; preserve | read-only | remove after parity |
|
||||
| `agents.status` | add workspace/lifecycle/runtime/roles; status remains presence | retain all legacy fields | lifecycle/roles authority; status may remain telemetry |
|
||||
| agent project/owner/prompt/tools/skills/config | preserve; validate tenant; derive typed capabilities without loss | N-1 reads | removal only by separate inventory |
|
||||
| fleet `backlog` | map to designated-project tasks; edges; claimed rows quarantine | freeze claims before switch; read-only compare | task/lease authority; retire after stabilization |
|
||||
|
||||
### 5.5 Required migration tests
|
||||
|
||||
Empty DB; exact production-shape snapshot; crash/resume; rollback before switch; N-1 startup/read/write; workspace/member negatives; status-shadow/no premature new status; `mission_tasks.status` write prohibition; tags/assignee/date/mission JSON/config/description/agent checksum; project congruence/current-milestone order; backlog freeze/no dispatch; and proof legacy declarations persist until contract release.
|
||||
|
||||
SI-001 adds frozen future executable evidence: empty and production-shape migrations create `missions_workspace_id_uidx` before either dependent FK; duplicate-key feasibility preflight returns no `(workspace_id,id)` duplicate groups without weakening global `id` uniqueness; N-1 startup/read/write behavior is unchanged; pre-switch rollback removes dependents before the candidate key; both exact FK column lists reconcile to the candidate key; and foreign-workspace mission references fail for both artifacts and approval decisions. TDD is not applicable to this design-only amendment; KBN-100 must implement these negative migration tests before runtime schema release.
|
||||
|
||||
Proposal-specific negatives must attempt: missing submission event, foreign-workspace submission event, foreign-workspace acceptance event, same-workspace event for another proposal, event for another target aggregate, and unrelated normal-command event. Every attempt must fail atomically with no accepted proposal and no target mutation.
|
||||
|
||||
## 6. Ownership and Coordinator split
|
||||
|
||||
coder2 solely owns `packages/db/src/schema.ts`, `packages/db/drizzle/**`, journal/metadata, and migration tests. No other lane generates migrations. Expand is additive; no drop/rename/narrow; constraints validate before NOT NULL; compatibility is same-DB only; contract is later.
|
||||
|
||||
KBN-200/coder4 owns pure `MechanicalCoordinatorDecisionEngineV1`: complete immutable snapshots in, deterministic eligibility/proposal/retry decisions out; no ID loading, SQL, Gateway, Valkey, proof, persistence, restart I/O, or LLM.
|
||||
|
||||
KBN-210/coder3 owns `MechanicalCoordinatorServicePortV1`: ID loading, locks, fresh proof, assignment/approval persistence, atomic fencing, lease/checkpoint/outbox, Valkey wakes, durable retry/quarantine, and `recoverFromPostgres`. Cycle: load snapshots → pure decision → persist assignment → authoritative approval/policy → acquire by IDs/locks → increment fence → lease → ack/heartbeat/checkpoint → submit to review or durable retry/quarantine. No completion/certification/merge method exists.
|
||||
|
||||
## 7. Exact Gateway/DTO freeze for KBN-105
|
||||
|
||||
### 7.1 Common wire rules
|
||||
|
||||
Base is `/api/v1/workspaces/:workspaceId`. Mutations require header `Idempotency-Key` (1–128 chars). Existing-aggregate mutations also require `If-Match-Version` (positive integer); create and privileged assignment-cycle requests are the only exceptions, while proposal submission carries `expectedTargetVersion` in its body. Body workspace fields are forbidden. Tenant denial follows one 404/403 policy without foreign existence detail.
|
||||
|
||||
```ts
|
||||
interface SuccessEnvelopeV1<T> {
|
||||
contractVersion: '1.0.0';
|
||||
data: T;
|
||||
aggregateRevision: string;
|
||||
correlationId: string;
|
||||
}
|
||||
interface ListEnvelopeV1<T> extends SuccessEnvelopeV1<T[]> {
|
||||
page: { cursor: string | null; nextCursor: string | null; limit: number };
|
||||
}
|
||||
```
|
||||
|
||||
Errors are the exact health/transport/version unions in §2 plus validation/auth/not-found. Public DTOs never expose/accept internal write proof.
|
||||
|
||||
### 7.2 Exact route registry
|
||||
|
||||
| Method/path | Request body/query | Success data |
|
||||
| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------- |
|
||||
| `GET /kanban-health` | none | `KanbanHealthResponseV1` |
|
||||
| `GET /projects` | `status,ownerUserId,ownerTeamId,cursor,limit` | project list |
|
||||
| `POST /projects` | `name,key,description,status,priority,ownerUserId XOR ownerTeamId,metadata` | project |
|
||||
| `GET /projects/:projectId` | none | project |
|
||||
| `PATCH /projects/:projectId` | editable create fields + expected header | project |
|
||||
| `POST /projects/:projectId/archive` | `reason` | project |
|
||||
| `GET /tasks` | `projectId,missionId,milestoneId,status,priority,ownerUserId,ownerTeamId,specialistRole,tag,dueState,archived,cursor,limit` | task summary list |
|
||||
| `POST /tasks` | `projectId,missionId?,milestoneId?,parentTaskId?,title,description?,acceptanceCriteria[],status,priority,rank,ownerUserId XOR ownerTeamId,specialistRole?,dueAt?,notBeforeAt?,estimateMinutes?,retryPolicy?,tagIds[],metadata` | task detail |
|
||||
| `GET /tasks/:taskId` | none | task detail including readiness/dependencies/assignment/lease/events |
|
||||
| `PATCH /tasks/:taskId` | editable non-transition fields | task detail |
|
||||
| `POST /tasks/:taskId/transition` | `toStatus,reason?` | task detail |
|
||||
| `POST /tasks/:taskId/move` | `toStatus?,beforeTaskId?,afterTaskId?` | task detail with persisted rank |
|
||||
| `POST /tasks/:taskId/archive` | `reason` | task detail |
|
||||
| `PUT /tasks/:taskId/tags` | `tagIds[]` | task detail |
|
||||
| `POST /tasks/:taskId/dependencies` | `predecessorTaskId,type` | dependency |
|
||||
| `DELETE /tasks/:taskId/dependencies/:predecessorTaskId` | no body | deleted dependency ID |
|
||||
| `GET /tasks/:taskId/events` | `cursor,limit` | event list |
|
||||
| `GET /tags` | `query,cursor,limit` | tag list |
|
||||
| `POST /tags` | `name,color?` | tag |
|
||||
| `GET /change-proposals` | `state,targetType,targetId,cursor,limit` | proposal list |
|
||||
| `POST /change-proposals` | `sourceNoteDigest,targetType,targetId,expectedTargetVersion,commandType,commandPayload` | inert proposal |
|
||||
| `GET /change-proposals/:proposalId` | none | proposal |
|
||||
| `POST /change-proposals/:proposalId/accept` | `reason` | proposal + normal command result |
|
||||
| `POST /change-proposals/:proposalId/reject` | `reason` | proposal |
|
||||
| `GET /coordinator/eligibility` | `projectId?,missionId?,cursor,limit` | `EligibilityDecisionV1[]` |
|
||||
| `POST /coordinator/assignment-cycles` | `limit` | assignment proposals; privileged internal |
|
||||
| `POST /coordinator/assignments/:assignmentId/approve` | `decision,reason,policyRevision,artifactIds[]` | approval decision |
|
||||
| `POST /coordinator/leases/acquire` | `taskId,assignmentId,approvalDecisionId,targetSessionId,leaseTtlSeconds` | lease with decimal-string fence |
|
||||
| `POST /coordinator/leases/:leaseId/ack` | `taskId,sessionId,fencingToken` | lease |
|
||||
| `POST /coordinator/leases/:leaseId/heartbeat` | `taskId,sessionId,fencingToken,extendSeconds` | lease |
|
||||
| `POST /coordinator/leases/:leaseId/checkpoints` | `taskId,sessionId,fencingToken,sequence,resumableSummary,artifactIds[],contextUsagePercent` | checkpoint |
|
||||
| `POST /coordinator/leases/:leaseId/submit-review` | `taskId,sessionId,fencingToken,artifactIds[],summary` | task in `in_review` |
|
||||
|
||||
All Coordinator mutations except human approval are service-identity-only. Generic task PATCH cannot perform claim/heartbeat/checkpoint/review/certification/completion shortcuts. Completion after certification uses a separately gated lifecycle command owned by the Portfolio/Sub-Orchestrator flow, not the Coordinator.
|
||||
|
||||
### 7.3 DTO invariants
|
||||
|
||||
Task summary/detail use exact schema vocabularies, owner union, `version: number`, `fencingCounter: string`, explicit `archivedAt/by/reason`, normalized tags, computed readiness, and separate assignment/lease. Assignment DTO includes one persisted ID, task/version, exact principal/agent/session, role, state, expiry, policy, proposer/reason. Lease/checkpoint DTOs serialize every fence as decimal string. Proposal DTO exposes no hidden write authority.
|
||||
|
||||
### 7.4 MCP ownership and mapping
|
||||
|
||||
coder3 exclusively owns:
|
||||
|
||||
- `apps/gateway/src/mcp/mcp.dto.ts`
|
||||
- `mcp.controller.ts`
|
||||
- `mcp.service.ts`
|
||||
- `mcp.module.ts`
|
||||
- `mcp.tokens.ts`
|
||||
- `mcp.service.spec.ts`
|
||||
|
||||
MCP tools are thin maps: `mosaic_projects_{list,get,create,update,archive}`, `mosaic_tasks_{list,get,create,update,transition,move,archive,set_tags,add_dependency,remove_dependency}`, and `mosaic_change_proposals_{list,get,submit,accept,reject}` to the exact routes above. coder4 owns CLI/projection clients only and must not edit Gateway MCP files.
|
||||
|
||||
KBN-105 publishes route+DTO fixture digest before KBN-110/120/130. Every web/CLI/MCP call must match this registry and the generated client.
|
||||
|
||||
## 8. Recovery contract and bounded delivery slice
|
||||
|
||||
Runtime must invoke normative `validateRecoveryPostureV1`; JSON Schema alone is insufficient. It rejects unknown fields, PITR/WAL mismatch, RPO better than mechanism, unsafe storage, and weakened High-assurance. High-assurance is RPO 15m/RTO 4h, WAL ≤5m, PITR ≥35d, base ≤24h, restore test ≤30d, break-glass ≤90d, encrypted separate-failure-domain storage.
|
||||
|
||||
KBN-115/coder2 owns `packages/config/src/recovery-posture.ts`, tests, and recovery runbook. It wires parser/refinement, override audit, mechanism assertions, restore test, and break-glass evidence. Any deployment manifest is separately enumerated and Mos-serialized. Recovery config has no SOT/gate/Coordinator authority fields.
|
||||
|
||||
## 9. Integration, security, and hold
|
||||
|
||||
Required release evidence includes KBN-101 foundation role/schema-boundary and post-KBN-100 real immutable-operation deployed-role certificates (not synthetic roles), empty/prod/partial/rollback/N-1 migration tests; cross-workspace and same-workspace wrong-project negatives; active-membership owners/principals; proposal inertness/normal acceptance; exact failure mapping; concurrent monotonic bigint fences; relational lease/checkpoint/evidence mismatch; immutability privileges/RESTRICT; recovery validation/mechanism evidence; endpoint registry alignment; accessible web journeys; author≠reviewer; mandatory SecReview; final Certifier pass/no merge authority.
|
||||
|
||||
### 9.1 SI-001 amendment gate and #757 boundary
|
||||
|
||||
- KBN-100 must provide the §5.2 candidate-key ordering, duplicate-feasibility, exact-FK reconciliation, empty/prod/N-1/rollback, and two-child foreign-workspace evidence before SI-001 can be certified closed.
|
||||
- All prior KCR-001–016 decisions and fixed SOT/tenant/authority, proposal-audit, approval, task-fencing, immutability, and no-cascade invariants remain unchanged.
|
||||
- Read-only PR #757 cross-check: its logical-agent connector lease/CAS fencing uses separate runtime tables/contracts and `lease_epoch`; rc.4 changes only the frozen `missions` candidate key. There is no shared table, index, FK, identity, fence, or authority semantic to consume or reconcile, and #757 remains owned by its existing lane.
|
||||
|
||||
The build hold remains active until independent re-review reports GO for KCR-001–016 and the rc.4 SI-001 amendment. Mos alone releases waves and serializes integration roots.
|
||||
@@ -0,0 +1,278 @@
|
||||
# Native Kanban/SOT P0–P3 — Dependency-Ordered Build Slices
|
||||
|
||||
**Status:** CANON INDEPENDENTLY APPROVED; PUBLICATION IN PROGRESS
|
||||
**Tracking:** [Mosaic Stack issue #751](https://git.mosaicstack.dev/mosaicstack/stack/issues/751)
|
||||
**Execution:** USC web1 only; collision-free GPT coder2/3/4/5 lanes
|
||||
**Contract:** `SHARED-CONTRACT.md` + four `contracts/*.v1.ts` files
|
||||
**Implementation hold:** no feature slice starts until the canon PR is merged to `main` with terminal-green CI; after merge, each slice remains held until every declared KBN prerequisite is complete.
|
||||
|
||||
> This publication file is not a runtime task authority. After cutover, repository `TASKS.md` is generated read-only and never imported.
|
||||
|
||||
## Execution invariants
|
||||
|
||||
- PostgreSQL is the sole writable SOT; current-main Drizzle is the persistence foundation.
|
||||
- Mutations require fresh internal PostgreSQL transaction-local write proof and fail closed otherwise.
|
||||
- Public health DTOs, Valkey, files, browser state, providers, and outage notes cannot authorize writes.
|
||||
- Outage notes return only through attributable `change_proposals`; proposal acceptance executes the normal command.
|
||||
- Mechanical Coordinator is non-LLM and cannot invent scope, waive gates, certify, or merge.
|
||||
- Certifier is final independent gate with no merge authority.
|
||||
- Workspace is the hard tenant. Project hierarchy is project-congruent. Assignment, approval, lease, fence, checkpoint, and evidence are relationally bound.
|
||||
- Recovery tiers change recovery posture only.
|
||||
|
||||
## 1. Collision-free ownership
|
||||
|
||||
| USC lane | Exclusive ownership | Must not edit |
|
||||
| ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- |
|
||||
| **coder2 — schema/recovery** | `packages/db/src/schema.ts`; `packages/db/drizzle/**`; DB tests; `packages/config/src/recovery-posture.ts`; `packages/config/src/recovery-posture.spec.ts`; `docs/runbooks/kanban-postgres-recovery.md` | Gateway, Brain repositories, Coordinator, web, CLI/importer |
|
||||
| **coder3 — domain/Gateway/MCP server** | Kanban repositories under `packages/brain/src/`; Gateway workspace/project/mission/milestone/task/kanban/health/coord modules; **exact MCP files:** `apps/gateway/src/mcp/mcp.dto.ts`, `mcp.controller.ts`, `mcp.service.ts`, `mcp.module.ts`, `mcp.tokens.ts`, `mcp.service.spec.ts`; Gateway root wiring/tests | DB schema/migrations, `packages/coord`, web, CLI/importer |
|
||||
| **coder4 — CLI → pure Coordinator → migration tooling** | In this one fixed lane order: KBN-120 (`packages/mosaic` CLI/projection) → KBN-200 (`packages/coord/src/mechanical/**`) → KBN-300/320 (`scripts/kanban-migration/**`) | DB, Gateway/MCP server, web |
|
||||
| **coder5 — web** | `apps/web/src/app/(dashboard)/{tasks,projects}/**`; `apps/web/src/components/{tasks,projects}/**`; Kanban web API/types; later Coordinator/migration-review routes | DB, Gateway, Coordinator, CLI/importer |
|
||||
| **Mos — publication/integration** | Contract amendments, exact endpoint registry publication, serialized root exports/manifests/lockfiles, integration gates | Active lane feature files |
|
||||
|
||||
Shared roots, package exports/manifests, lockfiles, and generated artifacts are integration-serialized. Contract changes stop affected lanes and require Mos approval.
|
||||
|
||||
## 2. Parallelization legend
|
||||
|
||||
- **SERIAL:** prerequisite must be complete and reviewed.
|
||||
- **PARALLEL-GROUP:** disjoint files and exact frozen contract permit concurrent work.
|
||||
- **LANE-SERIAL:** one lane's stated order cannot change.
|
||||
- **INTEGRATION-SERIAL:** component heads green first; semantic findings return to owner.
|
||||
|
||||
## 3. Corrected dependency graph
|
||||
|
||||
```text
|
||||
KBN-000 canon remediation
|
||||
-> KBN-010 threat/auth/constraint-impact gate (MUST COMPLETE)
|
||||
-> KBN-101 foundation role/schema-boundary certificate (SERIAL)
|
||||
-> KBN-100 schema + concrete N-1 migration implementation
|
||||
├─ KBN-101 post-KBN-100 deployed-role immutable-operation certificate (SERIAL)
|
||||
│ -> KBN-105 exact endpoint/DTO/error/registry freeze (SERIAL)
|
||||
│ ├─ KBN-110 domain + Gateway + MCP server implementation
|
||||
│ ├─ KBN-120 CLI/projection implementation [coder4 first]
|
||||
│ └─ KBN-130 web MVP implementation
|
||||
└─ KBN-115 recovery parser/mechanism slice [coder2 lane-serial]
|
||||
KBN-110 + KBN-120 + KBN-130 + KBN-115
|
||||
-> KBN-140 P1 integration/SIT
|
||||
-> KBN-200 pure decision engine [coder4 after KBN-120]
|
||||
-> KBN-210 persistence/service adapter + approval/lease binding
|
||||
-> KBN-220 Coordinator operations UI
|
||||
-> KBN-230 P2 concurrency/fault/gate integration
|
||||
KBN-230
|
||||
-> KBN-300 importer dry-run/apply/verify [coder4 after KBN-200]
|
||||
├─ KBN-310 migration reviewer UI
|
||||
└─ KBN-320 cutover/rollback tooling [coder4 after KBN-300]
|
||||
KBN-310 + KBN-320
|
||||
-> KBN-330 rehearsal/reconciliation
|
||||
-> KBN-340 owner-gated cutover/stabilization
|
||||
```
|
||||
|
||||
No consumer implementation begins before KBN-105. No schema work begins before KBN-010 completes and the KBN-101 foundation role/schema-boundary certificate passes; the real immutable-operation certificate follows KBN-100 and blocks KBN-105. The coder4 order is always KBN-120 → KBN-200 → KBN-300 → KBN-320.
|
||||
|
||||
## 4. P0 — Canon, threat gate, schema, and exact API freeze
|
||||
|
||||
### KBN-000 — Remediate and publish canon
|
||||
|
||||
- **Status:** COMPLETE — PR #752 squash-merged as `49e8a54`; issue #751 closed; post-merge pipeline #1798 terminal success.
|
||||
- **Owner:** Mosaic publication control plane.
|
||||
- **Mode:** SERIAL; completed.
|
||||
- **IN:** Resolve KCR-001–016 in requirements, schema, health, Coordinator, recovery, migration map, and slices; independent re-review.
|
||||
- **OUT:** Feature implementation.
|
||||
- **Depends on:** none.
|
||||
- **Contract surfaces:** all canon.
|
||||
- **Evidence:** strict TS; Prettier; per-finding traceability; independent author≠reviewer GO; Ultron GO; terminal-green CI.
|
||||
|
||||
### KBN-010 — Threat, authorization, and constraint-impact gate
|
||||
|
||||
- **Status:** IN PROGRESS — issue [#753](https://git.mosaicstack.dev/mosaicstack/stack/issues/753).
|
||||
- **Owner:** `kbn-coder3`; independent `secrev`.
|
||||
- **Mode:** SERIAL prerequisite of KBN-100.
|
||||
- **Exclusive files:** `docs/native-kanban-sot/KBN-010-THREAT-AUTH-CONSTRAINT-GATE.md` and task scratchpad only.
|
||||
- **IN:** Cross-workspace owners/principals/evidence; active membership; stale/forged health; approval forgery; fence monotonicity; audit retention; proposal target/audit-event forgery; service tokens; DB/Valkey outage.
|
||||
- **OUT:** Runtime/schema edits.
|
||||
- **Depends on:** KBN-000 independent re-review GO.
|
||||
- **Contract surfaces:** schema constraints, health proof, exact errors, command-family authorization.
|
||||
- **Evidence:** signed constraint-impact matrix; no unresolved schema-impact finding; SecReview pass.
|
||||
|
||||
### KBN-101 — PostgreSQL runtime/migration role split and deployed-role certification
|
||||
|
||||
- **Status:** IN PROGRESS — issue [#771](https://git.mosaicstack.dev/mosaicstack/stack/issues/771); rc.16 closes HIGH-1 current generic storage-wrapper authority: README/user-guide remove `storage migrate --run` guidance and false runner delegation; current source is direct-Drizzle, legacy N-1, uncertified, non-operative, and forbidden pending -02/-03/-06/-08 activation. The -06 fixture fails both exact former forms before inventory/status masking and source-consistency rejects current direct-Drizzle wrapper as runner delegation. It awaits independent exact-head re-review; implementation remains held.
|
||||
- **Owner:** Mos integration control plane; independently reviewed by security/Ultron.
|
||||
- **Mode:** SERIAL foundation certificate blocks KBN-100; its post-KBN-100 real immutable-operation certificate blocks KBN-105.
|
||||
- **IN:** Exact `DATABASE_URL` non-owner runtime versus `DATABASE_MIGRATION_URL` owner/migrator connection contract; sole published `mosaic-db-migrator --run|--verify` PostgreSQL DDL path and all legacy/future entrypoint closure; active migrate-tier destination only after runner prepare/verify through exact `--target-url-file /run/secrets/mosaic-migrate-target-url`, paired authenticated provider-version file, and signed `--target-attestation-file /run/mosaic-attestations/migrate-target.v1.json`; runner-only signing key/public-key isolation; canonical Vault KV-v2 target URL/version, generation-pinned renderer, importer CA/public-key/attestation plus privileged sealed producer-to-importer handoff, safe-fd/consumer-isolation/no-log-oracle, TLS/server/database/role/manifest/schema binding, expiry/replay/provider-rotation/TOCTOU/no-DML controls, and dedicated non-DDL importer; finite exact-path scanner/allowlist/active-route review plus unsuppressible automatic-startup/init/Compose-before-runner semantic negatives and every-path before-connect denial matrix; `DATABASE_TLS_CA_CERT_PATH` plus operator/IaC CA/server-key/cert lifecycle, exact service-DNS SANs, Vault/compose/Swarm mount modes, TLS server/bootstrap/rotation/rollback; PGlite exception; fixed two-int advisory lock; manifest-v1 logical-index/tag/exact-byte-SHA-256 ledger reconciliation including safe `0009`; fixed `mosaic` schema and exact `pg_catalog,mosaic` pooled session path; platform/schema/`NOLOGIN SUPERUSER` extension-owner/migrator/importer/runtime roles; approved-owner versus legacy-owner shadow pgvector transition; ownership, zero membership/no runtime secret, TEMP/ledger-read/default privilege and immutable grant proof; N-1 inactive prepared cards then atomic activation/rollback authority; Vault/redaction/observability/operator runbooks; one-card/one-PR implementation DAG.
|
||||
- **OUT:** Production mutation in this planning card; KBN-100 tables/data backfill; application API behavior; KBN-105 route/DTO freeze.
|
||||
- **Depends on:** KBN-010 completed.
|
||||
- **Contract surfaces:** [`KBN-101-DB-ROLE-SPLIT.md`](./KBN-101-DB-ROLE-SPLIT.md); `SHARED-CONTRACT.md` rc.15 amendment.
|
||||
- **Evidence:** foundation: exact `--help|--run|--verify`/exit/argv/import-negative plus DTO entrypoint negatives for every finite classified current DDL/static-bypass path (including `DATABASE_URL`-only, runner fixture, retired init, sanitized current operator guidance, both harness pairs, and `db:push` refusal); active migrate-tier paired URL/version/attestation files, signing/public-key isolation, canonical Vault KV-v2 authenticated version, generation-pinned renderer, importer CA, safe fd/TOCTOU/consumer-isolation/no-log-oracle, atomic JCS/Ed25519, digest/TLS/server/database/role/manifest/schema binding, expiry/replay/provider rotation/revocation, zero-connection versus zero-DML, prepared-target/importer/no-DDL negatives; clean/pre-0009/skipped/applied-late/duplicate/unknown/missing/corrupt/stale/backup plus public-to-`mosaic`/partial/reverse runner proof; fixed-lock contention/crash/readiness/unrelated-key tests; runtime cannot invoke migrations/DDL/TEMP; actual pgvector 0.8.2 control metadata, fresh/approved-owner existing/legacy-owner shadow/partial-resume-rollback/N-1 pgvector evidence with `rolcanlogin=false`, `rolsuper=true`, zero members, external-superuser `SET ROLE`/`RESET ROLE` audit, `pg_extension.extowner`, owner-bearing member/schema/version and runtime/migrator/schema-owner/importer/all-service-role `SET ROLE`/ALTER/DROP/member-update denial; disposable standalone, federated/Swarm, and two-gateway verified-TLS positives plus both-pair CA/SAN/downgrade/key mode/UID-GID/URL-secret consumer-isolation and legacy-drain/`hostssl` zero-plaintext negatives; exclusive bootstrap/renderer/manifest ownership test; catalog relocation/vector-query/operator/Drizzle-only-`mosaic`, role/grant/search-path/pool-reset/identifier checks; N-1/atomic TLS-only rollback/no-force-on-red rehearsal; named Vault/bootstrap-control-plane/CA-overlap/redaction/operator evidence; independent author≠reviewer security GO. Post-KBN-100: real deployed non-owner INSERT/SELECT and UPDATE/DELETE denial for immutable event/artifact/evidence relations plus Ultron GO.
|
||||
|
||||
### KBN-100 — Unified Drizzle schema and concrete N-1 migration
|
||||
|
||||
- **Owner:** **coder2**.
|
||||
- **Mode:** SERIAL.
|
||||
- **Exclusive files:** `packages/db/src/schema.ts`, `packages/db/drizzle/**`, DB tests.
|
||||
- **IN:** All frozen tables/joins/enums; workspace/project-congruent constraints; owners/principals; tags/archive; change proposals with both workspace-aware task-event composite FKs and frozen event-before-proposal DDL order; assignment approvals; durable execution/quarantine; monotonic bigint fence; exact checkpoint/evidence joins; RESTRICT/immutability; concrete current-main expand/backfill/switch/contract map.
|
||||
- **OUT:** Repositories, Gateway, Coordinator behavior, UI, importer.
|
||||
- **Depends on:** **KBN-010 completed and KBN-101 foundation role/schema-boundary certificate PASS**. KBN-100 is blocked until both are terminal; it rebases on KBN-101 main, restores generated Drizzle declaration/snapshot/journal consistency, and confines procedural immutable-table grant/trigger/backfill work to its schema ownership. Its new relations are then subject to KBN-101 post-KBN-100 deployed-role certification.
|
||||
- **Contract surfaces:** `kanban-schema.v1.ts`; SHARED-CONTRACT current-main delta map.
|
||||
- **Evidence:** reviewed SQL; empty/prod-shape/partial-resume/rollback tests; N-1 app safety; legacy columns remain declared; workspace/project mismatch negatives; proposal event-FK missing/foreign-workspace tests; one active lease; monotonic fence; parent-delete RESTRICT; immutability privileges; SecReview.
|
||||
|
||||
### KBN-105 — Exact Gateway/MCP endpoint, DTO, and error freeze
|
||||
|
||||
- **Owner:** Mos + coder3 contract author; independent endpoint-alignment reviewer.
|
||||
- **Mode:** SERIAL after KBN-100; prerequisite for KBN-110/120/130.
|
||||
- **Exclusive files:** canonical endpoint-registry/DTO contract docs; no implementation.
|
||||
- **IN:** Exact routes and methods from SHARED-CONTRACT §8; request/success/error fields; status codes; pagination/filter/revision envelopes; idempotency/expected-version headers/fields; proposal commands; health proof exclusion from public DTOs; MCP tool-to-route map.
|
||||
- **OUT:** Controller/service/client implementation.
|
||||
- **Depends on:** KBN-100 and KBN-101 post-KBN-100 deployed-role immutable-operation certification PASS.
|
||||
- **Contract surfaces:** health/error unions; schema IDs/statuses; Gateway DTO freeze.
|
||||
- **Evidence:** every FE/CLI/MCP call maps 1:1 to a route; 503/502-504/409 non-cross-map fixtures; contract digest published.
|
||||
|
||||
### KBN-115 — Recovery posture parser, mechanisms, and evidence
|
||||
|
||||
- **Owner:** **coder2**, lane-serial after KBN-100.
|
||||
- **Mode:** PARALLEL with KBN-110/120/130 after KBN-105.
|
||||
- **Exclusive files:** `packages/config/src/recovery-posture.ts`, `.spec.ts`, `docs/runbooks/kanban-postgres-recovery.md`; deployment-specific backup manifest changes are a separately enumerated Mos integration patch.
|
||||
- **IN:** Wire normative `validateRecoveryPostureV1`; override audit; backup/WAL/PITR mechanism assertions; off-cluster encryption/failure-domain checks; restore and break-glass evidence procedure.
|
||||
- **OUT:** SOT/gate/Coordinator policy knobs; DB business schema.
|
||||
- **Depends on:** KBN-100, KBN-105.
|
||||
- **Contract surfaces:** `recovery-posture.v1.ts` only.
|
||||
- **Evidence:** impossible-combination tests; High-assurance weakening tests; selected-tier mechanism verification; restore and break-glass evidence; SecReview.
|
||||
|
||||
## 5. P1 — Thin native MVP
|
||||
|
||||
### KBN-110 — Workspace-safe domain, Gateway, MCP server, and proposal commands
|
||||
|
||||
- **Owner:** **coder3**.
|
||||
- **Mode:** PARALLEL-GROUP P1-A after KBN-105.
|
||||
- **Exclusive files:** ownership map, including all exact MCP server files listed there.
|
||||
- **IN:** Workspace-safe repositories; project/task/dependency/tag/archive CRUD; transitions; exact owners; assignment/approval/link/artifact queries; submit/query/accept/reject change proposals; health endpoint; internal write-proof mint/revalidation; event/outbox atomicity; frozen DTOs/routes.
|
||||
- **OUT:** Scheduling algorithm, web, CLI, DB schema.
|
||||
- **Depends on:** KBN-100, KBN-105.
|
||||
- **Contract surfaces:** all four TypeScript contracts and exact registry.
|
||||
- **Evidence:** DTO/service/controller/integration tests; active-membership and no-oracle negatives; proposal cannot mutate directly; submission event is the new proposal's exact `change_proposal.submitted` event; acceptance links the executed normal command for the locked proposal and same workspace/target; missing, foreign-workspace, unrelated-proposal/target/command event negatives; exact failure mapping; endpoint registry; SecReview.
|
||||
|
||||
### KBN-120 — CLI, MCP client mapping, and generated projection
|
||||
|
||||
- **Owner:** **coder4**; first coder4 slice.
|
||||
- **Mode:** PARALLEL-GROUP P1-A after KBN-105.
|
||||
- **Exclusive files:** `packages/mosaic/src/commands/{kanban,tasks,projects}.ts`; `packages/mosaic/src/projections/**`; tests. **No `apps/gateway/src/mcp/**` edits.\*\*
|
||||
- **IN:** Frozen query/mutation routes; proposal commands; compact context; generated `TASKS.md`; deliberate denial/transport/conflict handling.
|
||||
- **OUT:** Gateway/MCP server, file importer, raw SQL/Valkey, Coordinator.
|
||||
- **Depends on:** KBN-105; runtime integration later requires KBN-110.
|
||||
- **Evidence:** contract fixtures; same revision; no import parser; same idempotency key on transport retry; 503 never auto-retried.
|
||||
|
||||
### KBN-130 — Writable Kanban/List and minimal Projects UI
|
||||
|
||||
- **Owner:** **coder5**.
|
||||
- **Mode:** PARALLEL-GROUP P1-A after KBN-105.
|
||||
- **Exclusive files:** web ownership map.
|
||||
- **IN:** Workspace context; projects; tasks; tags; explicit archive; detail; accessible move/reorder; filters; dependency/readiness; owner/assignment/lease; audit; proposal visibility; conflict/loading/error/reconnect.
|
||||
- **OUT:** Gateway/schema, Coordinator operations UI, migration UI.
|
||||
- **Depends on:** KBN-105; runtime integration later requires KBN-110.
|
||||
- **Evidence:** frozen contract mocks; real-Gateway journeys; keyboard/non-drag; tags/archive semantics; no-oracle tenant negatives; 503/transport/409 distinct UI.
|
||||
|
||||
### KBN-140 — P1 integration and situational gate
|
||||
|
||||
- **Owner:** Mos integration; independent reviewer/SecReview/Certifier.
|
||||
- **Mode:** INTEGRATION-SERIAL.
|
||||
- **IN:** KBN-110/120/130/115; unavoidable root exports only.
|
||||
- **OUT:** P2 behavior.
|
||||
- **Depends on:** KBN-110, KBN-120, KBN-130, KBN-115.
|
||||
- **Evidence:** clean migration; web/CLI/MCP/projection revision parity; forged/expired health negatives; change-proposal event-chain success plus missing/foreign/unrelated-event negatives; tag/archive; tenant negatives; endpoint registry; author-independent review; Certifier pass.
|
||||
|
||||
## 6. P2 — Mechanical Coordinator
|
||||
|
||||
### KBN-200 — Pure deterministic decision engine
|
||||
|
||||
- **Owner:** **coder4**; second coder4 slice, strictly after KBN-120.
|
||||
- **Mode:** SERIAL in coder4 lane.
|
||||
- **Exclusive files:** `packages/coord/src/mechanical/**` and pure tests.
|
||||
- **IN:** `MechanicalCoordinatorDecisionEngineV1`; complete immutable snapshots; eligibility/explanation; fairness/order; capability matching; expiry/retry/quarantine decisions.
|
||||
- **OUT:** ID loading, PostgreSQL, Drizzle, Gateway, Valkey, health-proof minting, persistence, `recoverFromPostgres`, LLM calls.
|
||||
- **Depends on:** KBN-140 (or Mos may release after KBN-120 + frozen types if no P1 semantic risk remains).
|
||||
- **Evidence:** deterministic/property tests; snapshot completeness; no I/O/model imports; no authority methods.
|
||||
|
||||
### KBN-210 — Coordinator persistence/service adapter and approval-bound leases
|
||||
|
||||
- **Owner:** **coder3**.
|
||||
- **Mode:** SERIAL after KBN-200.
|
||||
- **Exclusive files:** Gateway `coord` and repositories.
|
||||
- **IN:** `MechanicalCoordinatorServicePortV1`; snapshot loading; proposal persistence; manual/versioned policy approval; acquire by IDs; reload+lock task/assignment/approval/session; fresh txn-local write proof; atomic task fence increment; lease/ack/heartbeat/checkpoint/submit; durable retry/quarantine; outbox/Valkey wake; restart recovery.
|
||||
- **OUT:** Pure algorithm, UI, DB schema.
|
||||
- **Depends on:** KBN-110, KBN-200.
|
||||
- **Evidence:** forged/stale approval rejection; target/session/version/expiry/policy checks; concurrent monotonic fences; same-workspace mismatch negatives; bigint precision; stale worker rejection; DB/Valkey faults; SecReview.
|
||||
|
||||
### KBN-220 — Coordinator operations UI
|
||||
|
||||
- **Owner:** **coder5**.
|
||||
- **Mode:** after KBN-210 exact DTO freeze.
|
||||
- **IN:** Roster; eligibility; persisted assignment state; approvals/overrides; exact lease/fence; durable retry/quarantine; role/gate/Certifier visibility.
|
||||
- **OUT:** Scheduling decisions, schema, merge control for Certifier.
|
||||
- **Depends on:** KBN-210.
|
||||
- **Evidence:** authorized journeys; reason required; stale refresh; no Certifier merge; endpoint alignment/accessibility.
|
||||
|
||||
### KBN-230 — P2 concurrency/fault/gate integration
|
||||
|
||||
- **Owner:** Mos integration; independent reviewer/SecReview/Certifier.
|
||||
- **Mode:** INTEGRATION-SERIAL.
|
||||
- **Depends on:** KBN-200, KBN-210, KBN-220.
|
||||
- **Evidence:** one lease; monotonic fences; exact relational mismatches rejected; expired proof; forged healthy; approval binding; restart; durable quarantine; outbox recovery; author≠reviewer; Certifier final/no merge.
|
||||
|
||||
## 7. P3 — Shadow migration and cutover
|
||||
|
||||
### KBN-300 — One-way importer dry-run/apply/verify
|
||||
|
||||
- **Owner:** **coder4**; third coder4 slice.
|
||||
- **Mode:** after KBN-230.
|
||||
- **Exclusive files:** `scripts/kanban-migration/import/**`.
|
||||
- **IN:** Immutable jarvis-brain/Vikunja snapshots; deterministic mapping; source digest/lineage; Gateway writes; rejects; no dispatch.
|
||||
- **OUT:** Bidirectional sync, direct DB/file canonical writes, unrelated brain data.
|
||||
- **Depends on:** KBN-230.
|
||||
- **Evidence:** idempotency; counts/fields; malformed/foreign rejects; no dispatch; SecReview.
|
||||
|
||||
### KBN-310 — Shadow reviewer UI
|
||||
|
||||
- **Owner:** **coder5**.
|
||||
- **Mode:** PARALLEL-GROUP P3-A after KBN-300 report freeze.
|
||||
- **IN:** Read-only counts/diffs/rejects/lineage/sign-off.
|
||||
- **OUT:** Apply/cutover mutations.
|
||||
- **Depends on:** KBN-300.
|
||||
- **Evidence:** read-only and tenant tests; pagination/accessibility.
|
||||
|
||||
### KBN-320 — Cutover/rollback tooling
|
||||
|
||||
- **Owner:** **coder4**; fourth coder4 slice, after KBN-300.
|
||||
- **Mode:** PARALLEL-GROUP P3-A with KBN-310.
|
||||
- **Exclusive files:** `scripts/kanban-migration/cutover/**`.
|
||||
- **IN:** Freeze assertion; backup/checksum; final delta; client switch; legacy writer/credential shutdown; rollback delta; stabilization.
|
||||
- **OUT:** Destructive deletion, reverse sync, ungated production execution.
|
||||
- **Depends on:** KBN-300.
|
||||
- **Evidence:** fail-safe rehearsal; no dual writer; rollback authority; SecReview.
|
||||
|
||||
### KBN-330 — Migration rehearsal/reconciliation
|
||||
|
||||
- **Owner:** Mos + coder4 support + independent data reviewer.
|
||||
- **Mode:** INTEGRATION-SERIAL.
|
||||
- **Depends on:** KBN-310, KBN-320.
|
||||
- **Evidence:** signed exceptions; selected-tier restore; backlog hold; no legacy changes; Certifier readiness.
|
||||
|
||||
### KBN-340 — Final cutover/stabilization
|
||||
|
||||
- **Owner:** Mos/control plane; owner-gated operation.
|
||||
- **Mode:** SERIAL.
|
||||
- **Depends on:** KBN-330 PASS and Jason authorization.
|
||||
- **Evidence:** no legacy writer; scoped Gateway identities; no accidental dispatch; terminal green health/CI; Certifier evidence; owner retirement approval.
|
||||
|
||||
## 8. Consistent USC wave schedule
|
||||
|
||||
| Wave | coder2 | coder3 | coder4 | coder5 |
|
||||
| ---- | ----------------------------------------- | ------------------------------------------------------------------------------------ | ------------------------------ | ------------------------------ |
|
||||
| 0 | Wait | **KBN-010** | Wait | Wait |
|
||||
| 0.5 | Wait | **KBN-101 foundation** Mos-controlled role/connection contract and certificate | Wait | Wait |
|
||||
| 1 | **KBN-100** after KBN-101 foundation PASS | Review bounded schema/grant implementation | Wait | Wait |
|
||||
| 1.5 | Certification support | **KBN-101 post-KBN-100 deployed-role immutable-operation certificate**, then KBN-105 | Wait | Wait |
|
||||
| 2 | **KBN-115** after KBN-100 | **KBN-105** exact freeze, then KBN-110 | **KBN-120** only after KBN-105 | **KBN-130** only after KBN-105 |
|
||||
| 3 | Review support | Finish KBN-110 | **KBN-200 after KBN-120** | Finish KBN-130 |
|
||||
| 4 | — | **KBN-210 after KBN-200** | Review/support | **KBN-220 after KBN-210 DTOs** |
|
||||
| 5 | — | P2 remediation | **KBN-300 then KBN-320** | **KBN-310** |
|
||||
|
||||
Mos alone releases slices and lifts the build hold after independent re-review GO.
|
||||
@@ -0,0 +1,206 @@
|
||||
/**
|
||||
* Mosaic Native Kanban — frozen health/error contract v1.
|
||||
* Publication contract only; no runtime implementation is included here.
|
||||
*
|
||||
* PostgreSQL is the sole writable SOT. Public health DTOs are observations,
|
||||
* never write authority. Only an internal transaction-local proof produced by
|
||||
* the PostgreSQL adapter may authorize a mutation.
|
||||
*/
|
||||
|
||||
export const KANBAN_CONTRACT_VERSION = '1.0.0' as const;
|
||||
|
||||
export const kanbanHealthStates = ['healthy', 'read-only-degraded', 'write-unavailable'] as const;
|
||||
export type KanbanHealthState = (typeof kanbanHealthStates)[number];
|
||||
|
||||
interface KanbanHealthBaseV1 {
|
||||
contractVersion: typeof KANBAN_CONTRACT_VERSION;
|
||||
checkedAt: string;
|
||||
/** Observation expires at this RFC 3339 instant; it still never authorizes writes. */
|
||||
validUntil: string;
|
||||
policyRevision: string;
|
||||
reasons: string[];
|
||||
}
|
||||
|
||||
export interface HealthyKanbanHealthResponseV1 extends KanbanHealthBaseV1 {
|
||||
state: 'healthy';
|
||||
readHealthProven: true;
|
||||
writeHealthProven: true;
|
||||
}
|
||||
|
||||
export interface ReadOnlyDegradedKanbanHealthResponseV1 extends KanbanHealthBaseV1 {
|
||||
state: 'read-only-degraded';
|
||||
readHealthProven: true;
|
||||
writeHealthProven: false;
|
||||
}
|
||||
|
||||
export interface WriteUnavailableKanbanHealthResponseV1 extends KanbanHealthBaseV1 {
|
||||
state: 'write-unavailable';
|
||||
readHealthProven: false;
|
||||
writeHealthProven: false;
|
||||
}
|
||||
|
||||
/** Public, discriminated observation. Contradictory combinations are unrepresentable. */
|
||||
export type KanbanHealthResponseV1 =
|
||||
| HealthyKanbanHealthResponseV1
|
||||
| ReadOnlyDegradedKanbanHealthResponseV1
|
||||
| WriteUnavailableKanbanHealthResponseV1;
|
||||
|
||||
/** Pure evaluation context. It cannot authorize a mutation. */
|
||||
export interface KanbanEvaluationContextV1 {
|
||||
contractVersion: typeof KANBAN_CONTRACT_VERSION;
|
||||
workspaceId: string;
|
||||
correlationId: string;
|
||||
now: string;
|
||||
policyRevision: string;
|
||||
observedHealth: KanbanHealthResponseV1;
|
||||
}
|
||||
|
||||
/**
|
||||
* Non-exported brand: public DTO deserialization cannot construct this type.
|
||||
* The PostgreSQL adapter mints it only after a fresh write probe inside the same
|
||||
* transaction and validates checkedAt <= now < validUntil and policy revision.
|
||||
*/
|
||||
declare const postgresWriteHealthProofBrand: unique symbol;
|
||||
export interface PostgresWriteHealthProofV1 {
|
||||
readonly [postgresWriteHealthProofBrand]: true;
|
||||
readonly source: 'postgres-transaction-local-write-probe';
|
||||
readonly transactionId: string;
|
||||
readonly checkedAt: string;
|
||||
readonly validUntil: string;
|
||||
readonly policyRevision: string;
|
||||
}
|
||||
|
||||
/** Internal mutation context; MUST NOT appear in REST/MCP/CLI request DTOs. */
|
||||
export interface InternalKanbanMutationContextV1 {
|
||||
contractVersion: typeof KANBAN_CONTRACT_VERSION;
|
||||
workspaceId: string;
|
||||
correlationId: string;
|
||||
causationId?: string;
|
||||
idempotencyKey: string;
|
||||
now: string;
|
||||
expectedPolicyRevision: string;
|
||||
writeProof: PostgresWriteHealthProofV1;
|
||||
}
|
||||
|
||||
interface MutationFailureBaseV1 {
|
||||
contractVersion: typeof KANBAN_CONTRACT_VERSION;
|
||||
retryable: false;
|
||||
requestOutcome: 'not_applied';
|
||||
idempotencyKey: string;
|
||||
correlationId: string;
|
||||
message: string;
|
||||
}
|
||||
|
||||
/** KCR-016: code/state pairing is exact and cannot cross-map. */
|
||||
export interface ReadOnlyWriteHealthDenialV1 extends MutationFailureBaseV1 {
|
||||
kind: 'deliberate_fail_closed_denial';
|
||||
code: 'KANBAN_WRITE_HEALTH_UNPROVEN';
|
||||
healthState: 'read-only-degraded';
|
||||
checkedAt: string;
|
||||
}
|
||||
|
||||
export interface WriteUnavailableDenialV1 extends MutationFailureBaseV1 {
|
||||
kind: 'deliberate_fail_closed_denial';
|
||||
code: 'KANBAN_WRITE_UNAVAILABLE';
|
||||
healthState: 'write-unavailable';
|
||||
checkedAt: string;
|
||||
}
|
||||
|
||||
export type DeliberateWriteDenialV1 = ReadOnlyWriteHealthDenialV1 | WriteUnavailableDenialV1;
|
||||
|
||||
export const transportErrorCodes = [
|
||||
'GATEWAY_UNREACHABLE',
|
||||
'GATEWAY_TIMEOUT',
|
||||
'UPSTREAM_BAD_GATEWAY',
|
||||
] as const;
|
||||
export type TransportErrorCode = (typeof transportErrorCodes)[number];
|
||||
|
||||
/** Client-normalized transport uncertainty; never an authoritative 503 body. */
|
||||
export interface RetryableTransportErrorV1 {
|
||||
contractVersion: typeof KANBAN_CONTRACT_VERSION;
|
||||
kind: 'retryable_transport_error';
|
||||
code: TransportErrorCode;
|
||||
retryable: true;
|
||||
requestOutcome: 'unknown';
|
||||
/** Retry MUST reuse this exact key. */
|
||||
idempotencyKey: string;
|
||||
correlationId: string;
|
||||
message: string;
|
||||
}
|
||||
|
||||
export interface VersionConflictV1 {
|
||||
contractVersion: typeof KANBAN_CONTRACT_VERSION;
|
||||
kind: 'version_conflict';
|
||||
code: 'AGGREGATE_VERSION_CONFLICT';
|
||||
retryable: false;
|
||||
requestOutcome: 'not_applied';
|
||||
aggregateType: 'project' | 'mission' | 'milestone' | 'task' | 'change_proposal';
|
||||
aggregateId: string;
|
||||
expectedVersion: number;
|
||||
actualVersion: number;
|
||||
idempotencyKey: string;
|
||||
correlationId: string;
|
||||
message: string;
|
||||
}
|
||||
|
||||
export type KanbanMutationFailureV1 =
|
||||
| DeliberateWriteDenialV1
|
||||
| RetryableTransportErrorV1
|
||||
| VersionConflictV1;
|
||||
|
||||
export const kanbanHealthCapabilities: Readonly<
|
||||
Record<KanbanHealthState, { canonicalReads: boolean; mutations: boolean }>
|
||||
> = {
|
||||
healthy: { canonicalReads: true, mutations: true },
|
||||
'read-only-degraded': { canonicalReads: true, mutations: false },
|
||||
'write-unavailable': { canonicalReads: false, mutations: false },
|
||||
};
|
||||
|
||||
/** Exact HTTP/error normalization freeze; 503, transport, and 409 cannot cross-map. */
|
||||
export const kanbanFailureHttpMapV1 = {
|
||||
KANBAN_WRITE_HEALTH_UNPROVEN: {
|
||||
httpStatus: 503,
|
||||
kind: 'deliberate_fail_closed_denial',
|
||||
requestOutcome: 'not_applied',
|
||||
retryable: false,
|
||||
},
|
||||
KANBAN_WRITE_UNAVAILABLE: {
|
||||
httpStatus: 503,
|
||||
kind: 'deliberate_fail_closed_denial',
|
||||
requestOutcome: 'not_applied',
|
||||
retryable: false,
|
||||
},
|
||||
AGGREGATE_VERSION_CONFLICT: {
|
||||
httpStatus: 409,
|
||||
kind: 'version_conflict',
|
||||
requestOutcome: 'not_applied',
|
||||
retryable: false,
|
||||
},
|
||||
GATEWAY_UNREACHABLE: {
|
||||
httpStatus: 502,
|
||||
kind: 'retryable_transport_error',
|
||||
requestOutcome: 'unknown',
|
||||
retryable: true,
|
||||
},
|
||||
GATEWAY_TIMEOUT: {
|
||||
httpStatus: 504,
|
||||
kind: 'retryable_transport_error',
|
||||
requestOutcome: 'unknown',
|
||||
retryable: true,
|
||||
},
|
||||
UPSTREAM_BAD_GATEWAY: {
|
||||
httpStatus: 502,
|
||||
kind: 'retryable_transport_error',
|
||||
requestOutcome: 'unknown',
|
||||
retryable: true,
|
||||
},
|
||||
} as const;
|
||||
|
||||
/**
|
||||
* Required negative contract tests:
|
||||
* - contradictory state/proof booleans fail type/schema validation;
|
||||
* - expired internal proof and policy mismatch deny before mutation;
|
||||
* - Valkey-only liveness cannot mint PostgresWriteHealthProofV1;
|
||||
* - public/caller-forged `healthy` cannot enter InternalKanbanMutationContextV1;
|
||||
* - authoritative 503, transport 502/504/timeout, and 409 mappings are exhaustive.
|
||||
*/
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,419 @@
|
||||
/**
|
||||
* Mosaic Native Kanban — frozen Mechanical Coordinator contracts v1.
|
||||
*
|
||||
* The pure decision engine and persistence/orchestration service are separate.
|
||||
* Neither surface can create scope, edit acceptance, waive gates, certify,
|
||||
* merge, release a deployment, or close a provider issue.
|
||||
*/
|
||||
|
||||
import type {
|
||||
DeliberateWriteDenialV1,
|
||||
InternalKanbanMutationContextV1,
|
||||
KanbanEvaluationContextV1,
|
||||
KanbanMutationFailureV1,
|
||||
RetryableTransportErrorV1,
|
||||
VersionConflictV1,
|
||||
} from './health-state.v1.js';
|
||||
|
||||
export const COORDINATOR_CONTRACT_VERSION = '1.0.0' as const;
|
||||
export type Uuid = string;
|
||||
export type IsoTimestamp = string;
|
||||
/** PostgreSQL bigint-safe decimal string; never a JavaScript number. */
|
||||
export type FencingTokenV1 = string;
|
||||
|
||||
export const specialistRoles = [
|
||||
'planning',
|
||||
'enhance',
|
||||
'coder',
|
||||
'review',
|
||||
'security-review',
|
||||
'pr-monitor',
|
||||
'certifier',
|
||||
] as const;
|
||||
export type SpecialistRole = (typeof specialistRoles)[number];
|
||||
|
||||
/** One vocabulary shared with task_assignment_state_v1 in the Drizzle schema. */
|
||||
export const assignmentStates = [
|
||||
'awaiting_approval',
|
||||
'policy_pre_authorized',
|
||||
'approved',
|
||||
'rejected',
|
||||
'leased',
|
||||
'released',
|
||||
'expired',
|
||||
'superseded',
|
||||
] as const;
|
||||
export type AssignmentStateV1 = (typeof assignmentStates)[number];
|
||||
|
||||
export const readinessStates = [
|
||||
'dependency-gated',
|
||||
'schedule-gated',
|
||||
'policy-gated',
|
||||
'lease-available',
|
||||
'leased',
|
||||
'retry-delayed',
|
||||
'exhausted',
|
||||
'quarantined',
|
||||
] as const;
|
||||
export type ReadinessState = (typeof readinessStates)[number];
|
||||
|
||||
export interface RetryStateSnapshotV1 {
|
||||
disposition: 'available' | 'retry_delayed' | 'quarantined' | 'exhausted';
|
||||
attemptCount: number;
|
||||
maxAttempts: number;
|
||||
nextEligibleAt: IsoTimestamp | null;
|
||||
idempotent: boolean;
|
||||
terminalReason: string | null;
|
||||
version: number;
|
||||
}
|
||||
|
||||
export interface TaskEligibilitySnapshotV1 {
|
||||
workspaceId: Uuid;
|
||||
taskId: Uuid;
|
||||
taskVersion: number;
|
||||
projectId: Uuid;
|
||||
projectActive: boolean;
|
||||
missionId: Uuid | null;
|
||||
missionActive: boolean;
|
||||
status: 'ready';
|
||||
priority: 'critical' | 'high' | 'medium' | 'low';
|
||||
boardRank: string;
|
||||
dueAt: IsoTimestamp | null;
|
||||
notBeforeAt: IsoTimestamp | null;
|
||||
createdAt: IsoTimestamp;
|
||||
requiredRole: SpecialistRole;
|
||||
requiredCapabilities: readonly string[];
|
||||
blockingDependencies: readonly {
|
||||
taskId: Uuid;
|
||||
done: boolean;
|
||||
completionConditionSatisfied: boolean;
|
||||
}[];
|
||||
releaseApproval: {
|
||||
decisionId: Uuid;
|
||||
approved: boolean;
|
||||
policyRevision: string;
|
||||
} | null;
|
||||
activeLeaseId: Uuid | null;
|
||||
retry: RetryStateSnapshotV1;
|
||||
}
|
||||
|
||||
export interface AgentSessionSnapshotV1 {
|
||||
workspaceId: Uuid;
|
||||
agentId: Uuid;
|
||||
sessionId: Uuid;
|
||||
state: 'available' | 'busy';
|
||||
roles: readonly SpecialistRole[];
|
||||
capabilities: readonly string[];
|
||||
capacity: number;
|
||||
activeLeaseCount: number;
|
||||
heartbeatAt: IsoTimestamp;
|
||||
}
|
||||
|
||||
export interface EligibilityExplanationV1 {
|
||||
taskId: Uuid;
|
||||
eligible: boolean;
|
||||
readiness: ReadinessState;
|
||||
reasons: readonly {
|
||||
gate:
|
||||
| 'status'
|
||||
| 'project'
|
||||
| 'mission'
|
||||
| 'dependency'
|
||||
| 'schedule'
|
||||
| 'retry'
|
||||
| 'approval'
|
||||
| 'lease'
|
||||
| 'capability'
|
||||
| 'capacity'
|
||||
| 'health';
|
||||
satisfied: boolean;
|
||||
code: string;
|
||||
detail: string;
|
||||
}[];
|
||||
policyRevision: string;
|
||||
evaluatedAt: IsoTimestamp;
|
||||
}
|
||||
|
||||
export interface AssignmentProposalDecisionV1 {
|
||||
workspaceId: Uuid;
|
||||
taskId: Uuid;
|
||||
taskVersion: number;
|
||||
targetAgentId: Uuid;
|
||||
targetSessionId: Uuid;
|
||||
specialistRole: SpecialistRole;
|
||||
initialState: 'awaiting_approval' | 'policy_pre_authorized';
|
||||
policyRevision: string;
|
||||
explanation: EligibilityExplanationV1;
|
||||
expiresAt: IsoTimestamp;
|
||||
}
|
||||
|
||||
export interface AssignmentCycleSnapshotV1 {
|
||||
context: KanbanEvaluationContextV1;
|
||||
tasks: readonly TaskEligibilitySnapshotV1[];
|
||||
sessions: readonly AgentSessionSnapshotV1[];
|
||||
workspaceFairness: Readonly<Record<Uuid, number>>;
|
||||
limit: number;
|
||||
}
|
||||
|
||||
export interface AssignmentCycleDecisionV1 {
|
||||
evaluatedTaskCount: number;
|
||||
proposals: readonly AssignmentProposalDecisionV1[];
|
||||
explanations: readonly EligibilityExplanationV1[];
|
||||
}
|
||||
|
||||
export interface LeaseExpirySnapshotV1 {
|
||||
workspaceId: Uuid;
|
||||
taskId: Uuid;
|
||||
taskVersion: number;
|
||||
leaseId: Uuid;
|
||||
assignmentId: Uuid;
|
||||
sessionId: Uuid;
|
||||
fencingToken: FencingTokenV1;
|
||||
state: 'pending_ack' | 'active';
|
||||
acknowledgeBy: IsoTimestamp;
|
||||
expiresAt: IsoTimestamp;
|
||||
lastHeartbeatAt: IsoTimestamp | null;
|
||||
retry: RetryStateSnapshotV1;
|
||||
}
|
||||
|
||||
export interface LeaseExpiryDecisionV1 {
|
||||
leaseId: Uuid;
|
||||
action: 'retain' | 'release' | 'retry' | 'quarantine' | 'exhaust';
|
||||
reason: string;
|
||||
nextEligibleAt: IsoTimestamp | null;
|
||||
}
|
||||
|
||||
/** Pure package owned by KBN-200. It receives complete immutable snapshots. */
|
||||
export interface MechanicalCoordinatorDecisionEngineV1 {
|
||||
evaluateAssignmentCycle(snapshot: AssignmentCycleSnapshotV1): AssignmentCycleDecisionV1;
|
||||
explainEligibility(
|
||||
context: KanbanEvaluationContextV1,
|
||||
task: TaskEligibilitySnapshotV1,
|
||||
sessions: readonly AgentSessionSnapshotV1[],
|
||||
): EligibilityExplanationV1;
|
||||
decideLeaseExpiry(
|
||||
context: KanbanEvaluationContextV1,
|
||||
lease: LeaseExpirySnapshotV1,
|
||||
): LeaseExpiryDecisionV1;
|
||||
}
|
||||
|
||||
export interface PersistedAssignmentV1 {
|
||||
assignmentId: Uuid;
|
||||
workspaceId: Uuid;
|
||||
taskId: Uuid;
|
||||
taskVersion: number;
|
||||
targetAgentId: Uuid;
|
||||
targetSessionId: Uuid;
|
||||
specialistRole: SpecialistRole;
|
||||
state: AssignmentStateV1;
|
||||
policyRevision: string;
|
||||
proposedBy: { kind: 'user' | 'agent'; id: Uuid };
|
||||
reason: string;
|
||||
createdAt: IsoTimestamp;
|
||||
expiresAt: IsoTimestamp;
|
||||
}
|
||||
|
||||
export interface TaskLeaseV1 {
|
||||
leaseId: Uuid;
|
||||
workspaceId: Uuid;
|
||||
taskId: Uuid;
|
||||
taskVersion: number;
|
||||
assignmentId: Uuid;
|
||||
agentId: Uuid;
|
||||
sessionId: Uuid;
|
||||
state: 'pending_ack' | 'active';
|
||||
fencingToken: FencingTokenV1;
|
||||
attempt: number;
|
||||
acquiredAt: IsoTimestamp;
|
||||
acknowledgeBy: IsoTimestamp;
|
||||
lastHeartbeatAt: IsoTimestamp | null;
|
||||
expiresAt: IsoTimestamp;
|
||||
}
|
||||
|
||||
interface ServiceCommandBaseV1 {
|
||||
context: InternalKanbanMutationContextV1;
|
||||
taskId: Uuid;
|
||||
expectedTaskVersion: number;
|
||||
}
|
||||
|
||||
export interface AcquireApprovedLeaseCommandV1 extends ServiceCommandBaseV1 {
|
||||
assignmentId: Uuid;
|
||||
approvalDecisionId: Uuid;
|
||||
targetSessionId: Uuid;
|
||||
leaseTtlSeconds: number;
|
||||
}
|
||||
|
||||
export interface LeaseCommandV1 extends ServiceCommandBaseV1 {
|
||||
leaseId: Uuid;
|
||||
sessionId: Uuid;
|
||||
fencingToken: FencingTokenV1;
|
||||
}
|
||||
|
||||
export interface HeartbeatLeaseCommandV1 extends LeaseCommandV1 {
|
||||
extendSeconds: number;
|
||||
}
|
||||
|
||||
export interface CheckpointCommandV1 extends LeaseCommandV1 {
|
||||
sequence: number;
|
||||
resumableSummary: string;
|
||||
artifactIds: readonly Uuid[];
|
||||
contextUsagePercent: number;
|
||||
}
|
||||
|
||||
export interface SubmitForReviewCommandV1 extends LeaseCommandV1 {
|
||||
artifactIds: readonly Uuid[];
|
||||
summary: string;
|
||||
}
|
||||
|
||||
export interface ReleaseLeaseCommandV1 extends LeaseCommandV1 {
|
||||
reason:
|
||||
| 'worker_requested'
|
||||
| 'ack_timeout'
|
||||
| 'heartbeat_timeout'
|
||||
| 'task_submitted'
|
||||
| 'policy_revoked'
|
||||
| 'shutdown';
|
||||
}
|
||||
|
||||
export interface AssignmentCycleCommandV1 {
|
||||
context: InternalKanbanMutationContextV1;
|
||||
limit: number;
|
||||
}
|
||||
|
||||
export interface ExpirySweepCommandV1 {
|
||||
context: InternalKanbanMutationContextV1;
|
||||
limit: number;
|
||||
}
|
||||
|
||||
export interface RecoverCoordinatorCommandV1 {
|
||||
context: InternalKanbanMutationContextV1;
|
||||
}
|
||||
|
||||
interface CoordinatorRejectionBaseV1 {
|
||||
kind: 'coordinator_rejection';
|
||||
retryable: false;
|
||||
requestOutcome: 'not_applied';
|
||||
correlationId: Uuid;
|
||||
idempotencyKey: string;
|
||||
message: string;
|
||||
}
|
||||
|
||||
export type CoordinatorPolicyRejectionV1 =
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'WORKSPACE_MISMATCH' })
|
||||
| (CoordinatorRejectionBaseV1 & {
|
||||
code: 'TASK_NOT_ELIGIBLE';
|
||||
explanation: EligibilityExplanationV1;
|
||||
})
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'APPROVAL_REQUIRED' })
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'APPROVAL_STALE' })
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'ASSIGNMENT_STALE' })
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'ASSIGNMENT_TARGET_MISMATCH' })
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'POLICY_REVISION_MISMATCH' })
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'ARTIFACT_WORKSPACE_MISMATCH' })
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'LEASE_ALREADY_ACTIVE' })
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'LEASE_NOT_FOUND' })
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'LEASE_NOT_ACTIVE' })
|
||||
| (CoordinatorRejectionBaseV1 & {
|
||||
code: 'ACK_DEADLINE_EXPIRED';
|
||||
expiredAt: IsoTimestamp;
|
||||
})
|
||||
| (CoordinatorRejectionBaseV1 & {
|
||||
code: 'FENCING_TOKEN_STALE';
|
||||
currentFencingToken: FencingTokenV1;
|
||||
})
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'SESSION_MISMATCH' })
|
||||
| (CoordinatorRejectionBaseV1 & {
|
||||
code: 'HEARTBEAT_EXPIRED';
|
||||
expiredAt: IsoTimestamp;
|
||||
})
|
||||
| (CoordinatorRejectionBaseV1 & {
|
||||
code: 'CHECKPOINT_SEQUENCE_CONFLICT';
|
||||
currentSequence: number;
|
||||
})
|
||||
| (CoordinatorRejectionBaseV1 & { code: 'RETRY_EXHAUSTED' })
|
||||
| (CoordinatorRejectionBaseV1 & {
|
||||
code: 'NON_IDEMPOTENT_RETRY_REQUIRES_ORCHESTRATOR';
|
||||
});
|
||||
|
||||
/** Explicit mapping to the Gateway mutation failure union; no arbitrary booleans. */
|
||||
export type CoordinatorFailureV1 =
|
||||
| DeliberateWriteDenialV1
|
||||
| VersionConflictV1
|
||||
| RetryableTransportErrorV1
|
||||
| CoordinatorPolicyRejectionV1;
|
||||
|
||||
export interface CoordinatorSuccessV1<T> {
|
||||
ok: true;
|
||||
value: T;
|
||||
correlationId: Uuid;
|
||||
}
|
||||
export interface CoordinatorFailureResultV1 {
|
||||
ok: false;
|
||||
failure: CoordinatorFailureV1;
|
||||
}
|
||||
export type CoordinatorResultV1<T> = CoordinatorSuccessV1<T> | CoordinatorFailureResultV1;
|
||||
|
||||
export interface ExpirySweepResultV1 {
|
||||
examined: number;
|
||||
released: readonly Uuid[];
|
||||
retryScheduled: readonly Uuid[];
|
||||
quarantined: readonly Uuid[];
|
||||
exhausted: readonly Uuid[];
|
||||
}
|
||||
|
||||
export interface RestartRecoveryResultV1 {
|
||||
activeLeaseIds: readonly Uuid[];
|
||||
expiredLeaseIds: readonly Uuid[];
|
||||
pendingAssignmentIds: readonly Uuid[];
|
||||
pendingOutboxEventIds: readonly Uuid[];
|
||||
}
|
||||
|
||||
/** Persistence/Gateway adapter owned by KBN-210. */
|
||||
export interface MechanicalCoordinatorServicePortV1 {
|
||||
/** Loads immutable snapshots, invokes pure engine, and persists proposals atomically. */
|
||||
runAssignmentCycle(
|
||||
command: AssignmentCycleCommandV1,
|
||||
): Promise<CoordinatorResultV1<{ assignments: readonly PersistedAssignmentV1[] }>>;
|
||||
|
||||
/** Query path loads by ID; public health observation cannot authorize mutation. */
|
||||
getEligibilityExplanation(
|
||||
context: KanbanEvaluationContextV1,
|
||||
taskId: Uuid,
|
||||
): Promise<CoordinatorResultV1<EligibilityExplanationV1>>;
|
||||
|
||||
/**
|
||||
* Accepts IDs only. Implementation reloads and locks assignment + approval +
|
||||
* task + target session in PostgreSQL, then verifies workspace, task version,
|
||||
* target agent/session, state, expiry, policy revision, and current approval.
|
||||
*/
|
||||
acquireApprovedLease(
|
||||
command: AcquireApprovedLeaseCommandV1,
|
||||
): Promise<CoordinatorResultV1<TaskLeaseV1>>;
|
||||
|
||||
acknowledgeLease(command: LeaseCommandV1): Promise<CoordinatorResultV1<TaskLeaseV1>>;
|
||||
heartbeatLease(command: HeartbeatLeaseCommandV1): Promise<CoordinatorResultV1<TaskLeaseV1>>;
|
||||
appendCheckpoint(
|
||||
command: CheckpointCommandV1,
|
||||
): Promise<CoordinatorResultV1<{ checkpointId: Uuid }>>;
|
||||
submitForReview(
|
||||
command: SubmitForReviewCommandV1,
|
||||
): Promise<CoordinatorResultV1<{ taskVersion: number; status: 'in_review' }>>;
|
||||
releaseLease(command: ReleaseLeaseCommandV1): Promise<CoordinatorResultV1<{ released: true }>>;
|
||||
expireAndRecover(
|
||||
command: ExpirySweepCommandV1,
|
||||
): Promise<CoordinatorResultV1<ExpirySweepResultV1>>;
|
||||
recoverFromPostgres(
|
||||
command: RecoverCoordinatorCommandV1,
|
||||
): Promise<CoordinatorResultV1<RestartRecoveryResultV1>>;
|
||||
}
|
||||
|
||||
/** Compile-time mapping guarantee: Coordinator Gateway failures are Kanban failures or exact policy rejections. */
|
||||
export function isKanbanMutationFailureV1(
|
||||
failure: CoordinatorFailureV1,
|
||||
): failure is KanbanMutationFailureV1 {
|
||||
return (
|
||||
failure.kind === 'deliberate_fail_closed_denial' ||
|
||||
failure.kind === 'retryable_transport_error' ||
|
||||
failure.kind === 'version_conflict'
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,369 @@
|
||||
/**
|
||||
* Mosaic Native Kanban — frozen recovery-posture contract v1.
|
||||
* Recovery posture is configurable; SOT, write-health, Coordinator authority,
|
||||
* and gate semantics are not fields and cannot be overridden.
|
||||
*/
|
||||
|
||||
export const RECOVERY_POSTURE_CONTRACT_VERSION = '1.0.0' as const;
|
||||
export const recoveryTiers = ['lite', 'standard', 'high-assurance'] as const;
|
||||
export type RecoveryTier = (typeof recoveryTiers)[number];
|
||||
|
||||
export interface OffClusterStorageV1 {
|
||||
required: true;
|
||||
encrypted: true;
|
||||
separateFailureDomain: true;
|
||||
minimumCopies: number;
|
||||
storageClass: 'encrypted-object-storage' | 'encrypted-backup-target';
|
||||
}
|
||||
|
||||
export interface RecoveryPostureV1 {
|
||||
contractVersion: typeof RECOVERY_POSTURE_CONTRACT_VERSION;
|
||||
tier: RecoveryTier;
|
||||
targetRpoMinutes: number;
|
||||
targetRtoMinutes: number;
|
||||
baseBackupIntervalHours: number;
|
||||
/** null means WAL archival/PITR is disabled. */
|
||||
walArchiveIntervalMinutes: number | null;
|
||||
/** 0 means PITR is disabled. */
|
||||
pitrRetentionDays: number;
|
||||
restoreTestIntervalDays: number;
|
||||
breakGlassDrillIntervalDays: number;
|
||||
offClusterStorage: OffClusterStorageV1;
|
||||
}
|
||||
|
||||
export const recoveryPostureDefaults: Readonly<Record<RecoveryTier, RecoveryPostureV1>> = {
|
||||
lite: {
|
||||
contractVersion: RECOVERY_POSTURE_CONTRACT_VERSION,
|
||||
tier: 'lite',
|
||||
targetRpoMinutes: 24 * 60,
|
||||
targetRtoMinutes: 24 * 60,
|
||||
baseBackupIntervalHours: 24,
|
||||
walArchiveIntervalMinutes: null,
|
||||
pitrRetentionDays: 0,
|
||||
restoreTestIntervalDays: 90,
|
||||
breakGlassDrillIntervalDays: 365,
|
||||
offClusterStorage: {
|
||||
required: true,
|
||||
encrypted: true,
|
||||
separateFailureDomain: true,
|
||||
minimumCopies: 1,
|
||||
storageClass: 'encrypted-backup-target',
|
||||
},
|
||||
},
|
||||
standard: {
|
||||
contractVersion: RECOVERY_POSTURE_CONTRACT_VERSION,
|
||||
tier: 'standard',
|
||||
targetRpoMinutes: 60,
|
||||
targetRtoMinutes: 8 * 60,
|
||||
baseBackupIntervalHours: 24,
|
||||
walArchiveIntervalMinutes: 15,
|
||||
pitrRetentionDays: 14,
|
||||
restoreTestIntervalDays: 90,
|
||||
breakGlassDrillIntervalDays: 180,
|
||||
offClusterStorage: {
|
||||
required: true,
|
||||
encrypted: true,
|
||||
separateFailureDomain: true,
|
||||
minimumCopies: 1,
|
||||
storageClass: 'encrypted-object-storage',
|
||||
},
|
||||
},
|
||||
'high-assurance': {
|
||||
contractVersion: RECOVERY_POSTURE_CONTRACT_VERSION,
|
||||
tier: 'high-assurance',
|
||||
targetRpoMinutes: 15,
|
||||
targetRtoMinutes: 4 * 60,
|
||||
baseBackupIntervalHours: 24,
|
||||
walArchiveIntervalMinutes: 5,
|
||||
pitrRetentionDays: 35,
|
||||
restoreTestIntervalDays: 30,
|
||||
breakGlassDrillIntervalDays: 90,
|
||||
offClusterStorage: {
|
||||
required: true,
|
||||
encrypted: true,
|
||||
separateFailureDomain: true,
|
||||
minimumCopies: 1,
|
||||
storageClass: 'encrypted-object-storage',
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
/** Shape schema. Normative cross-field semantics are enforced by validateRecoveryPostureV1. */
|
||||
export const recoveryPostureJsonSchemaV1 = {
|
||||
$id: 'https://mosaicstack.dev/contracts/recovery-posture.v1.schema.json',
|
||||
$schema: 'https://json-schema.org/draft/2020-12/schema',
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
required: [
|
||||
'contractVersion',
|
||||
'tier',
|
||||
'targetRpoMinutes',
|
||||
'targetRtoMinutes',
|
||||
'baseBackupIntervalHours',
|
||||
'walArchiveIntervalMinutes',
|
||||
'pitrRetentionDays',
|
||||
'restoreTestIntervalDays',
|
||||
'breakGlassDrillIntervalDays',
|
||||
'offClusterStorage',
|
||||
],
|
||||
properties: {
|
||||
contractVersion: { const: RECOVERY_POSTURE_CONTRACT_VERSION },
|
||||
tier: { enum: recoveryTiers },
|
||||
targetRpoMinutes: { type: 'integer', minimum: 1 },
|
||||
targetRtoMinutes: { type: 'integer', minimum: 1 },
|
||||
baseBackupIntervalHours: { type: 'integer', minimum: 1 },
|
||||
walArchiveIntervalMinutes: {
|
||||
anyOf: [{ type: 'integer', minimum: 1 }, { type: 'null' }],
|
||||
},
|
||||
pitrRetentionDays: { type: 'integer', minimum: 0 },
|
||||
restoreTestIntervalDays: { type: 'integer', minimum: 1 },
|
||||
breakGlassDrillIntervalDays: { type: 'integer', minimum: 1 },
|
||||
offClusterStorage: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
required: ['required', 'encrypted', 'separateFailureDomain', 'minimumCopies', 'storageClass'],
|
||||
properties: {
|
||||
required: { const: true },
|
||||
encrypted: { const: true },
|
||||
separateFailureDomain: { const: true },
|
||||
minimumCopies: { type: 'integer', minimum: 1 },
|
||||
storageClass: {
|
||||
enum: ['encrypted-object-storage', 'encrypted-backup-target'],
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
} as const;
|
||||
|
||||
export const recoveryValidationCodes = [
|
||||
'INVALID_SHAPE',
|
||||
'UNKNOWN_FIELD',
|
||||
'PITR_REQUIRES_WAL',
|
||||
'WAL_REQUIRES_PITR',
|
||||
'RPO_BETTER_THAN_MECHANISM',
|
||||
'OFF_CLUSTER_REQUIRED',
|
||||
'HIGH_ASSURANCE_WEAKENED',
|
||||
] as const;
|
||||
export type RecoveryValidationCode = (typeof recoveryValidationCodes)[number];
|
||||
|
||||
export interface RecoveryValidationIssueV1 {
|
||||
code: RecoveryValidationCode;
|
||||
path: string;
|
||||
message: string;
|
||||
}
|
||||
export type RecoveryValidationResultV1 =
|
||||
| { ok: true; value: RecoveryPostureV1 }
|
||||
| { ok: false; issues: RecoveryValidationIssueV1[] };
|
||||
|
||||
const topLevelFields = new Set([
|
||||
'contractVersion',
|
||||
'tier',
|
||||
'targetRpoMinutes',
|
||||
'targetRtoMinutes',
|
||||
'baseBackupIntervalHours',
|
||||
'walArchiveIntervalMinutes',
|
||||
'pitrRetentionDays',
|
||||
'restoreTestIntervalDays',
|
||||
'breakGlassDrillIntervalDays',
|
||||
'offClusterStorage',
|
||||
]);
|
||||
const storageFields = new Set([
|
||||
'required',
|
||||
'encrypted',
|
||||
'separateFailureDomain',
|
||||
'minimumCopies',
|
||||
'storageClass',
|
||||
]);
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === 'object' && value !== null && !Array.isArray(value);
|
||||
}
|
||||
function isPositiveInteger(value: unknown): value is number {
|
||||
return Number.isInteger(value) && Number(value) > 0;
|
||||
}
|
||||
function isNonnegativeInteger(value: unknown): value is number {
|
||||
return Number.isInteger(value) && Number(value) >= 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Normative parser/refinement. Deployment code MUST call this function (or a
|
||||
* byte-for-byte behaviorally equivalent generated validator), not JSON Schema
|
||||
* shape validation alone.
|
||||
*/
|
||||
export function validateRecoveryPostureV1(input: unknown): RecoveryValidationResultV1 {
|
||||
const issues: RecoveryValidationIssueV1[] = [];
|
||||
if (!isRecord(input)) {
|
||||
return {
|
||||
ok: false,
|
||||
issues: [{ code: 'INVALID_SHAPE', path: '$', message: 'posture must be an object' }],
|
||||
};
|
||||
}
|
||||
|
||||
for (const key of Object.keys(input)) {
|
||||
if (!topLevelFields.has(key)) {
|
||||
issues.push({ code: 'UNKNOWN_FIELD', path: `$.${key}`, message: 'unknown field' });
|
||||
}
|
||||
}
|
||||
|
||||
const tier = input['tier'];
|
||||
const storage = input['offClusterStorage'];
|
||||
const integerFields = [
|
||||
'targetRpoMinutes',
|
||||
'targetRtoMinutes',
|
||||
'baseBackupIntervalHours',
|
||||
'restoreTestIntervalDays',
|
||||
'breakGlassDrillIntervalDays',
|
||||
] as const;
|
||||
|
||||
if (input['contractVersion'] !== RECOVERY_POSTURE_CONTRACT_VERSION) {
|
||||
issues.push({
|
||||
code: 'INVALID_SHAPE',
|
||||
path: '$.contractVersion',
|
||||
message: `must equal ${RECOVERY_POSTURE_CONTRACT_VERSION}`,
|
||||
});
|
||||
}
|
||||
if (!recoveryTiers.includes(tier as RecoveryTier)) {
|
||||
issues.push({ code: 'INVALID_SHAPE', path: '$.tier', message: 'unknown recovery tier' });
|
||||
}
|
||||
for (const field of integerFields) {
|
||||
if (!isPositiveInteger(input[field])) {
|
||||
issues.push({
|
||||
code: 'INVALID_SHAPE',
|
||||
path: `$.${field}`,
|
||||
message: 'must be a positive integer',
|
||||
});
|
||||
}
|
||||
}
|
||||
if (!isNonnegativeInteger(input['pitrRetentionDays'])) {
|
||||
issues.push({
|
||||
code: 'INVALID_SHAPE',
|
||||
path: '$.pitrRetentionDays',
|
||||
message: 'must be a nonnegative integer',
|
||||
});
|
||||
}
|
||||
if (
|
||||
input['walArchiveIntervalMinutes'] !== null &&
|
||||
!isPositiveInteger(input['walArchiveIntervalMinutes'])
|
||||
) {
|
||||
issues.push({
|
||||
code: 'INVALID_SHAPE',
|
||||
path: '$.walArchiveIntervalMinutes',
|
||||
message: 'must be null or a positive integer',
|
||||
});
|
||||
}
|
||||
|
||||
if (!isRecord(storage)) {
|
||||
issues.push({
|
||||
code: 'INVALID_SHAPE',
|
||||
path: '$.offClusterStorage',
|
||||
message: 'must be an object',
|
||||
});
|
||||
} else {
|
||||
for (const key of Object.keys(storage)) {
|
||||
if (!storageFields.has(key)) {
|
||||
issues.push({
|
||||
code: 'UNKNOWN_FIELD',
|
||||
path: `$.offClusterStorage.${key}`,
|
||||
message: 'unknown field',
|
||||
});
|
||||
}
|
||||
}
|
||||
if (
|
||||
storage['required'] !== true ||
|
||||
storage['encrypted'] !== true ||
|
||||
storage['separateFailureDomain'] !== true
|
||||
) {
|
||||
issues.push({
|
||||
code: 'OFF_CLUSTER_REQUIRED',
|
||||
path: '$.offClusterStorage',
|
||||
message: 'storage must be required, encrypted, and in a separate failure domain',
|
||||
});
|
||||
}
|
||||
if (!isPositiveInteger(storage['minimumCopies'])) {
|
||||
issues.push({
|
||||
code: 'INVALID_SHAPE',
|
||||
path: '$.offClusterStorage.minimumCopies',
|
||||
message: 'must be a positive integer',
|
||||
});
|
||||
}
|
||||
if (
|
||||
storage['storageClass'] !== 'encrypted-object-storage' &&
|
||||
storage['storageClass'] !== 'encrypted-backup-target'
|
||||
) {
|
||||
issues.push({
|
||||
code: 'INVALID_SHAPE',
|
||||
path: '$.offClusterStorage.storageClass',
|
||||
message: 'unsupported storage class',
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const wal = input['walArchiveIntervalMinutes'];
|
||||
const pitr = input['pitrRetentionDays'];
|
||||
if (pitr !== 0 && wal === null) {
|
||||
issues.push({
|
||||
code: 'PITR_REQUIRES_WAL',
|
||||
path: '$.pitrRetentionDays',
|
||||
message: 'PITR retention requires WAL archival',
|
||||
});
|
||||
}
|
||||
if (wal !== null && pitr === 0) {
|
||||
issues.push({
|
||||
code: 'WAL_REQUIRES_PITR',
|
||||
path: '$.walArchiveIntervalMinutes',
|
||||
message: 'WAL archival requires positive PITR retention',
|
||||
});
|
||||
}
|
||||
|
||||
if (
|
||||
isPositiveInteger(input['targetRpoMinutes']) &&
|
||||
isPositiveInteger(input['baseBackupIntervalHours']) &&
|
||||
(wal === null || isPositiveInteger(wal))
|
||||
) {
|
||||
const mechanismMinutes = wal === null ? input['baseBackupIntervalHours'] * 60 : wal;
|
||||
if (mechanismMinutes > input['targetRpoMinutes']) {
|
||||
issues.push({
|
||||
code: 'RPO_BETTER_THAN_MECHANISM',
|
||||
path: '$.targetRpoMinutes',
|
||||
message: `configured mechanism can only support ${mechanismMinutes} minutes`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (tier === 'high-assurance') {
|
||||
const weakened =
|
||||
!isPositiveInteger(input['targetRpoMinutes']) ||
|
||||
input['targetRpoMinutes'] > 15 ||
|
||||
!isPositiveInteger(input['targetRtoMinutes']) ||
|
||||
input['targetRtoMinutes'] > 4 * 60 ||
|
||||
!isPositiveInteger(input['baseBackupIntervalHours']) ||
|
||||
input['baseBackupIntervalHours'] > 24 ||
|
||||
!isPositiveInteger(wal) ||
|
||||
wal > 5 ||
|
||||
!isNonnegativeInteger(pitr) ||
|
||||
pitr < 35 ||
|
||||
!isPositiveInteger(input['restoreTestIntervalDays']) ||
|
||||
input['restoreTestIntervalDays'] > 30 ||
|
||||
!isPositiveInteger(input['breakGlassDrillIntervalDays']) ||
|
||||
input['breakGlassDrillIntervalDays'] > 90;
|
||||
if (weakened) {
|
||||
issues.push({
|
||||
code: 'HIGH_ASSURANCE_WEAKENED',
|
||||
path: '$',
|
||||
message: 'high-assurance posture may be strengthened but not weakened',
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (issues.length > 0) return { ok: false, issues };
|
||||
return { ok: true, value: input as unknown as RecoveryPostureV1 };
|
||||
}
|
||||
|
||||
export interface RecoveryPostureOverrideAuditV1 {
|
||||
actorId: string;
|
||||
reason: string;
|
||||
effectiveAt: string;
|
||||
policyRevision: string;
|
||||
previous: RecoveryPostureV1;
|
||||
next: RecoveryPostureV1;
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"extends": "../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"noEmit": true,
|
||||
"incremental": false,
|
||||
"declaration": false,
|
||||
"declarationMap": false,
|
||||
"sourceMap": false,
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"drizzle-orm": ["../../packages/db/node_modules/drizzle-orm/index.d.ts"],
|
||||
"drizzle-orm/pg-core": ["../../packages/db/node_modules/drizzle-orm/pg-core/index.d.ts"]
|
||||
}
|
||||
},
|
||||
"include": ["contracts/*.ts"]
|
||||
}
|
||||
@@ -0,0 +1,98 @@
|
||||
# Gateway Security Hardening Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Finish the requested gateway security hardening fixes in the existing `fix/gateway-security` worktree and produce a PR-ready branch.
|
||||
|
||||
**Architecture:** Tighten NestJS gateway boundaries in-place by enforcing auth guards, session validation, ownership checks, DTO validation, and Fastify security defaults. Preserve the current module structure and existing ESM import conventions.
|
||||
|
||||
**Tech Stack:** NestJS 11, Fastify, Socket.IO, Better Auth, class-validator, Vitest, pnpm, TypeScript ESM
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Reconcile Security Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `apps/gateway/src/chat/__tests__/chat-security.test.ts`
|
||||
- Modify: `apps/gateway/src/__tests__/resource-ownership.test.ts`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
- Encode the requested DTO constraints and socket-auth contract exactly.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `pnpm --filter @mosaicstack/gateway test -- src/chat/__tests__/chat-security.test.ts src/__tests__/resource-ownership.test.ts`
|
||||
|
||||
Expected: FAIL on current DTO/helper mismatch.
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
- Update DTO/helper/controller code only where tests prove a gap.
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run the same command and require green.
|
||||
|
||||
### Task 2: Align Gateway Runtime Hardening
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `apps/gateway/src/conversations/conversations.dto.ts`
|
||||
- Modify: `apps/gateway/src/chat/chat.dto.ts`
|
||||
- Modify: `apps/gateway/src/chat/chat.gateway-auth.ts`
|
||||
- Modify: `apps/gateway/src/chat/chat.gateway.ts`
|
||||
- Modify: `apps/gateway/src/main.ts`
|
||||
- Modify: `apps/gateway/src/app.module.ts`
|
||||
|
||||
**Step 1: Verify remaining requested deltas**
|
||||
|
||||
- Confirm code matches requested guard, rate limit, helmet, body limit, env validation, and CORS settings.
|
||||
|
||||
**Step 2: Apply minimal patch**
|
||||
|
||||
- Keep changes scoped to requested behavior only.
|
||||
|
||||
**Step 3: Run targeted tests**
|
||||
|
||||
Run: `pnpm --filter @mosaicstack/gateway test -- src/chat/__tests__/chat-security.test.ts src/__tests__/resource-ownership.test.ts`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 3: Verification, Review, and Delivery
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `docs/reports/code-review/gateway-security-20260313.md`
|
||||
- Create: `docs/reports/qa/gateway-security-20260313.md`
|
||||
- Modify: `docs/scratchpads/gateway-security-20260313.md`
|
||||
|
||||
**Step 1: Run baseline gates**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
pnpm typecheck
|
||||
pnpm lint
|
||||
```
|
||||
|
||||
**Step 2: Perform manual code review**
|
||||
|
||||
- Record correctness/security/testing/doc findings.
|
||||
|
||||
**Step 3: Commit and publish**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "fix(gateway): security hardening — auth guards, ownership checks, validation, rate limiting"
|
||||
git push origin fix/gateway-security
|
||||
```
|
||||
|
||||
**Step 4: Open PR and notify**
|
||||
|
||||
- Open PR titled `fix(gateway): security hardening — auth guards, ownership checks, validation, rate limiting`
|
||||
- Run `openclaw system event --text "PR ready: mosaic-mono-v1 fix/gateway-security — 7 security fixes" --mode now`
|
||||
- Remove worktree after PR is created.
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user