From 85a93428eb081748313f7319b8791f5677827689 Mon Sep 17 00:00:00 2001 From: Jason Woltje Date: Mon, 22 Jun 2026 03:11:31 -0500 Subject: [PATCH] =?UTF-8?q?feat(fleet):=20F4=20Phase=201=20=E2=80=94=20orc?= =?UTF-8?q?hestrator=20chat=20connector=20abstraction=20+=20Matrix=20desig?= =?UTF-8?q?n=20(#616)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From the north-star (#613) orchestrator-chat-connector decision: make the orchestrator's chat channel a pluggable, user-chosen connector (tmux | discord | matrix peers) and add Matrix (local homeserver) — starting with the small uniform abstraction so connectors drop in without touching fleet core. Phase 1 (this PR — design + scaffold): - src/fleet/connectors/types.ts: OrchestratorConnector interface (send / subscribe / health per the Lead's spec) + message/config types; thread-aware via optional threadId so Matrix threads + the future first-party Mosaic Discord plugin fit without an interface change. DEFAULT_CONNECTOR_KIND=tmux. - src/fleet/connectors/registry.ts: extensible factory registry; resolveConnectorKind defaults tmux (back-compat); createConnector throws ConnectorNotImplementedError until Phase 2 registers factories. - roster.schema.json: optional connector block (tmux|discord|matrix; matrix homeserver/user/room). Secrets via env (gateway pattern), never the roster. - docs/fleet/f4-matrix-connector.md: interface, config, Matrix client-server API mapping, Conduit-default local homeserver, phasing, back-compat. Self-contained connectors/ module — NO fleet.ts changes, so independent of the in-flight stacked fleet-config PR (#615). Verified: 7 connector tests green; tsc/eslint/prettier/sanitize clean; schema valid JSON. Refs #616, #613 Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01EsgTQzV5YUGk1JtCLP4B83 --- docs/TASKS.md | 4 + docs/fleet/f4-matrix-connector.md | 92 +++++++++++++++ docs/scratchpads/f4-matrix-connector.md | 19 +++ .../mosaic/framework/fleet/roster.schema.json | 29 +++++ .../src/fleet/connectors/registry.spec.ts | 85 ++++++++++++++ .../mosaic/src/fleet/connectors/registry.ts | 76 ++++++++++++ packages/mosaic/src/fleet/connectors/types.ts | 111 ++++++++++++++++++ 7 files changed, 416 insertions(+) create mode 100644 docs/fleet/f4-matrix-connector.md create mode 100644 docs/scratchpads/f4-matrix-connector.md create mode 100644 packages/mosaic/src/fleet/connectors/registry.spec.ts create mode 100644 packages/mosaic/src/fleet/connectors/registry.ts create mode 100644 packages/mosaic/src/fleet/connectors/types.ts diff --git a/docs/TASKS.md b/docs/TASKS.md index e9dda35..10f83e6 100644 --- a/docs/TASKS.md +++ b/docs/TASKS.md @@ -58,3 +58,7 @@ Active workstream is **W1 — Federation v1**. Workers should: ## F3-m3 — mosaic update re-seeds framework + relaunches agents (#609) — feat/f3-m3-update-reseed - Status: implemented + tested. Closes R13: `mosaic update` now re-seeds the framework (data-safe MOSAIC_SYNC_ONLY) after the CLI install so shipped launcher/runtime changes activate; `--relaunch` restarts rostered agents; `--no-reseed` opts out. Detail: scratchpads/f3-m3-update-reseed.md. + +## F4 — Orchestrator chat connector + Matrix (#616) — feat/f4-matrix-connector + +- Status: Phase 1 done (abstraction + scaffold). Connector interface (send/subscribe/health) + registry + roster connector schema + design doc; tmux default/back-compat; matrix/discord factories are Phase 2. 7 tests green; no fleet.ts changes (independent of #615). Detail: scratchpads/f4-matrix-connector.md. diff --git a/docs/fleet/f4-matrix-connector.md b/docs/fleet/f4-matrix-connector.md new file mode 100644 index 0000000..031192c --- /dev/null +++ b/docs/fleet/f4-matrix-connector.md @@ -0,0 +1,92 @@ +# F4 — Orchestrator chat connector + Matrix (local homeserver) + +> **Issue:** #616 · **Doctrine:** `docs/fleet/north-star.md` (#613) — orchestrator-chat-connector decision. +> **Status:** Phase 1 (abstraction + scaffold) in this PR; Phase 2+ are follow-ups (below). + +## Goal + +The fleet **orchestrator** is the operator's single point of contact. The north-star makes the +chat channel a **user-chosen connector** — tmux today, Discord live ("Mos"), with Matrix / +Telegram / Slack configurable. F4 adds **Matrix** (local homeserver) as a **peer** connector and, +first, the small **connector abstraction** that makes connectors pluggable without touching fleet +core. + +## The abstraction (Phase 1 — this PR) + +Connectors implement one small, uniform interface (`src/fleet/connectors/types.ts`): + +```ts +interface OrchestratorConnector { + readonly kind: 'tmux' | 'discord' | 'matrix'; + send(message: OutboundMessage): Promise; // orchestrator → human + subscribe(handler: (m: InboundMessage) => void): Unsubscribe; // human → orchestrator + health(): Promise; // reachable + authenticated +} +``` + +- **send / subscribe / health** — the only surface fleet core depends on. `SendResult` is the + ack half; `health()` is the liveness half. +- **Thread-aware by metadata** — `OutboundMessage.threadId` / `InboundMessage.threadId` are + optional, so thread-capable connectors (Matrix rooms/threads, the future first-party Mosaic + Discord plugin) fit **without an interface change**. +- **Registry** (`registry.ts`) — implementations register a factory by kind; `createConnector(config)` + resolves one from roster config. Phase 1 ships the registry + `resolveConnectorKind` (defaults + `tmux` when a roster declares no connector — **back-compat**); the factories land in Phase 2. + +### Config model + +A roster may carry an optional `connector` block (`roster.schema.json`); absent ⇒ tmux. + +```yaml +connector: + kind: matrix # tmux | discord | matrix + matrix: + homeserver_url: https://matrix.example.internal + user_id: '@mos:example.internal' + room_id: '!abc:example.internal' +``` + +**Secrets are never in the roster.** `MATRIX_ACCESS_TOKEN` / `DISCORD_BOT_TOKEN` come from the +environment (the gateway env-config pattern that already masks them). The sanitization gate would +reject a token committed to a shipped file anyway. + +## Matrix connector (Phase 2) + +The connector speaks the **Matrix client-server API** directly over HTTPS (`fetch` — no SDK needed +for MVP), so it is **homeserver-agnostic**: + +| Op | Matrix CS-API | +| ----------- | ------------------------------------------------------------------------ | +| `send` | `PUT /_matrix/client/v3/rooms/{roomId}/send/m.room.message/{txnId}` | +| `subscribe` | `GET /_matrix/client/v3/sync` (long-poll, `since` token) → room timeline | +| `health` | `GET /_matrix/client/versions` (reachable) + `…/account/whoami` (authed) | +| threads | `m.thread` relations ↔ `threadId` | + +## Local homeserver (infra, not connector code) + +Strategic default: a **self-hosted** homeserver on our own infra — no third-party gateway. + +- **Default: Conduit** (Rust, single binary, low resource) — trivial to stand up for a fleet/dev + homeserver. +- **Alternative: Synapse** (mature, feature-complete) for scale. + +The connector only needs `homeserver_url` + `user_id` + `room_id` + an access token, so the +homeserver choice is a **deployment** concern (a Phase-2 deploy guide), not connector code. + +## Phasing + +| Phase | Scope | This PR | +| ----- | --------------------------------------------------------------------------------------- | ------- | +| **1** | Connector interface + types, registry + kind resolution, roster `connector` schema, doc | ✅ yes | +| 2 | Matrix CS-API client (fetch-based send/sync/health) + registered factory + tests | follow | +| 2 | `fleet init` / `configure` connector-selection UX; roster parse wires the block | follow | +| 2 | systemd launch wiring so the orchestrator starts on the chosen connector | follow | +| 3 | Conduit deploy guide; first-party Mosaic Discord (threads) registers as a connector | follow | + +## Back-compat & boundaries + +- Existing rosters (no `connector`) resolve to tmux — **zero change**. +- Fleet core never branches on connector kind; it depends only on the interface. +- Cross-host reach rides the **federation** layer (W1), not a bespoke broker (north-star assumption). +- Phase 1 touches **no** `fleet.ts` core (a self-contained `connectors/` module), so it is + independent of the in-flight fleet-config PRs. diff --git a/docs/scratchpads/f4-matrix-connector.md b/docs/scratchpads/f4-matrix-connector.md new file mode 100644 index 0000000..290875e --- /dev/null +++ b/docs/scratchpads/f4-matrix-connector.md @@ -0,0 +1,19 @@ +# F4 — Orchestrator chat connector + Matrix (#616) + +- **Issue:** #616 · **Branch:** `feat/f4-matrix-connector` (off main; independent of #615) · **Doctrine:** north-star #613. + +## Phase 1 (this PR) — abstraction + scaffold + +- `src/fleet/connectors/types.ts`: `OrchestratorConnector` (send/subscribe/health) + message/config types; thread-aware via optional `threadId`; `DEFAULT_CONNECTOR_KIND=tmux`. +- `src/fleet/connectors/registry.ts`: extensible factory registry; `resolveConnectorKind` (defaults tmux, back-compat); `createConnector` throws `ConnectorNotImplementedError` until Phase 2 registers factories. +- `roster.schema.json`: optional `connector` block (tmux|discord|matrix; matrix homeserver/user/room; secrets via env, never roster). +- Design doc `docs/fleet/f4-matrix-connector.md`: interface, config, Matrix CS-API mapping, Conduit-default infra, phasing. +- **No fleet.ts changes** → self-contained, zero conflict with stacked #615. + +## Verification + +- 7 connector tests green; tsc/eslint/prettier/sanitize clean; schema valid JSON. + +## Phase 2+ (follow-ups, in the doc) + +Matrix CS-API client (fetch send/sync/health) + factory; init/configure connector-selection UX + roster-parse wiring; systemd launch wiring; Conduit deploy guide; first-party Mosaic Discord (threads) as a connector. diff --git a/packages/mosaic/framework/fleet/roster.schema.json b/packages/mosaic/framework/fleet/roster.schema.json index 08ba105..f23d4e0 100644 --- a/packages/mosaic/framework/fleet/roster.schema.json +++ b/packages/mosaic/framework/fleet/roster.schema.json @@ -113,6 +113,35 @@ } } } + }, + "connector": { + "description": "Orchestrator chat connector (F4). Optional — absent means tmux (back-compat). Secrets (access/bot tokens) come from the environment, never this file.", + "type": "object", + "additionalProperties": false, + "required": ["kind"], + "properties": { + "kind": { + "enum": ["tmux", "discord", "matrix"] + }, + "matrix": { + "type": "object", + "additionalProperties": false, + "required": ["homeserver_url", "user_id", "room_id"], + "properties": { + "homeserver_url": { "type": "string" }, + "user_id": { "type": "string" }, + "room_id": { "type": "string" } + } + }, + "discord": { + "type": "object", + "additionalProperties": false, + "required": ["channel_id"], + "properties": { + "channel_id": { "type": "string" } + } + } + } } } } diff --git a/packages/mosaic/src/fleet/connectors/registry.spec.ts b/packages/mosaic/src/fleet/connectors/registry.spec.ts new file mode 100644 index 0000000..4fcd1b4 --- /dev/null +++ b/packages/mosaic/src/fleet/connectors/registry.spec.ts @@ -0,0 +1,85 @@ +import { describe, it, expect, beforeEach } from 'vitest'; +import { + KNOWN_CONNECTOR_KINDS, + isKnownConnectorKind, + resolveConnectorKind, + registerConnector, + hasConnector, + createConnector, + ConnectorNotImplementedError, + _resetConnectorRegistry, +} from './registry.js'; +import type { ConnectorConfig, OrchestratorConnector } from './types.js'; + +function fakeConnector(kind: 'tmux' | 'discord' | 'matrix'): OrchestratorConnector { + return { + kind, + send: async () => ({ delivered: true, messageId: 'x' }), + subscribe: () => () => {}, + health: async () => ({ reachable: true, authenticated: true }), + }; +} + +describe('connector registry (F4 Phase 1)', () => { + beforeEach(() => { + _resetConnectorRegistry(); + }); + + it('knows the three peer connector kinds', () => { + expect(KNOWN_CONNECTOR_KINDS).toEqual(['tmux', 'discord', 'matrix']); + }); + + it('isKnownConnectorKind guards correctly', () => { + expect(isKnownConnectorKind('matrix')).toBe(true); + expect(isKnownConnectorKind('irc')).toBe(false); + expect(isKnownConnectorKind(42)).toBe(false); + }); + + it('resolveConnectorKind defaults to tmux when config is absent (back-compat)', () => { + expect(resolveConnectorKind(undefined)).toBe('tmux'); + expect(resolveConnectorKind({ kind: 'matrix' })).toBe('matrix'); + }); + + it('createConnector throws ConnectorNotImplementedError for an unregistered kind', () => { + const cfg: ConnectorConfig = { kind: 'matrix' }; + expect(() => createConnector(cfg)).toThrow(ConnectorNotImplementedError); + expect(() => createConnector(cfg)).toThrow(/not implemented yet/i); + }); + + it('createConnector with no config resolves the default kind (tmux) and reports it unimplemented in Phase 1', () => { + try { + createConnector(); + throw new Error('expected throw'); + } catch (err) { + expect(err).toBeInstanceOf(ConnectorNotImplementedError); + expect((err as ConnectorNotImplementedError).kind).toBe('tmux'); + } + }); + + it('register → has → create resolves a registered factory', () => { + expect(hasConnector('matrix')).toBe(false); + registerConnector('matrix', (cfg) => fakeConnector(cfg.kind)); + expect(hasConnector('matrix')).toBe(true); + + const connector = createConnector({ kind: 'matrix' }); + expect(connector.kind).toBe('matrix'); + }); + + it('passes the config through to the factory', () => { + let received: ConnectorConfig | null = null; + registerConnector('matrix', (cfg) => { + received = cfg; + return fakeConnector(cfg.kind); + }); + const cfg: ConnectorConfig = { + kind: 'matrix', + matrix: { + homeserverUrl: 'https://matrix.internal', + userId: '@mos:internal', + roomId: '!room:internal', + }, + }; + createConnector(cfg); + expect(received).toEqual(cfg); + }); +}); diff --git a/packages/mosaic/src/fleet/connectors/registry.ts b/packages/mosaic/src/fleet/connectors/registry.ts new file mode 100644 index 0000000..36a8fcc --- /dev/null +++ b/packages/mosaic/src/fleet/connectors/registry.ts @@ -0,0 +1,76 @@ +/** + * Connector registry (F4 Phase 1). + * + * A tiny extensible registry so connector implementations (Phase 2: tmux, + * Discord, Matrix) register a factory by kind and fleet core resolves one from + * roster config without branching on kind. Phase 1 ships the registry + the + * config→kind resolution; the connector factories land in Phase 2. + */ + +import { + type ConnectorConfig, + type ConnectorKind, + type OrchestratorConnector, + DEFAULT_CONNECTOR_KIND, +} from './types.js'; + +/** The set of connector kinds the framework recognizes. */ +export const KNOWN_CONNECTOR_KINDS: readonly ConnectorKind[] = ['tmux', 'discord', 'matrix']; + +/** Type guard: is `value` a known connector kind? */ +export function isKnownConnectorKind(value: unknown): value is ConnectorKind { + return typeof value === 'string' && (KNOWN_CONNECTOR_KINDS as readonly string[]).includes(value); +} + +/** + * Resolve the connector kind from roster config. Absent config ⇒ the default + * (tmux) so existing rosters keep working unchanged (back-compat). + */ +export function resolveConnectorKind(config?: ConnectorConfig): ConnectorKind { + return config?.kind ?? DEFAULT_CONNECTOR_KIND; +} + +/** A factory builds a live connector from its validated config. */ +export type ConnectorFactory = (config: ConnectorConfig) => OrchestratorConnector; + +/** Thrown when no factory is registered for a requested kind. */ +export class ConnectorNotImplementedError extends Error { + constructor(public readonly kind: ConnectorKind) { + super( + `Connector "${kind}" is not implemented yet. ` + + `Register a factory via registerConnector('${kind}', …) (F4 Phase 2).`, + ); + this.name = 'ConnectorNotImplementedError'; + } +} + +const registry = new Map(); + +/** Register a connector factory for a kind (idempotent — last registration wins). */ +export function registerConnector(kind: ConnectorKind, factory: ConnectorFactory): void { + registry.set(kind, factory); +} + +/** True when a factory is registered for `kind`. */ +export function hasConnector(kind: ConnectorKind): boolean { + return registry.has(kind); +} + +/** + * Build a connector from roster config. Throws `ConnectorNotImplementedError` + * when no factory is registered for the resolved kind (the Phase-1 default for + * every kind until Phase 2 registers them). + */ +export function createConnector(config?: ConnectorConfig): OrchestratorConnector { + const kind = resolveConnectorKind(config); + const factory = registry.get(kind); + if (!factory) { + throw new ConnectorNotImplementedError(kind); + } + return factory(config ?? { kind }); +} + +/** Test/runtime helper: drop all registrations. */ +export function _resetConnectorRegistry(): void { + registry.clear(); +} diff --git a/packages/mosaic/src/fleet/connectors/types.ts b/packages/mosaic/src/fleet/connectors/types.ts new file mode 100644 index 0000000..da44605 --- /dev/null +++ b/packages/mosaic/src/fleet/connectors/types.ts @@ -0,0 +1,111 @@ +/** + * Orchestrator chat connectors (F4). + * + * A connector mediates the chat channel between the fleet **orchestrator** and + * its human operator. Connectors are PEERS — tmux (default), Discord, Matrix, + * and future first-party plugins — selected per fleet, never hardwired. Fleet + * core depends only on the small uniform interface below, so a new connector + * drops in without touching the fleet. + * + * The interface is deliberately minimal: send (orchestrator → human), + * subscribe (human → orchestrator), health (reachable/authed liveness). Thread + * support is optional metadata (`threadId`) so thread-capable connectors + * (Matrix rooms/threads, the future Mosaic Discord plugin) fit without an + * interface change. + */ + +/** The connector kinds shipped/known to the framework. */ +export type ConnectorKind = 'tmux' | 'discord' | 'matrix'; + +/** A message the orchestrator sends out to the human operator. */ +export interface OutboundMessage { + /** Message body (markdown where the connector supports it). */ + text: string; + /** Optional thread/topic id for thread-capable connectors. */ + threadId?: string; + /** Optional attachment references (paths or URLs); connector-dependent. */ + attachments?: string[]; +} + +/** A message received from the human operator. */ +export interface InboundMessage { + /** Message body. */ + text: string; + /** Thread/topic id if the connector carries one. */ + threadId?: string; + /** Opaque sender identifier (connector-scoped). */ + sender: string; + /** ISO-8601 timestamp the connector assigns/observes. */ + ts: string; +} + +/** Result of a send — the "ack" half of ack/health. */ +export interface SendResult { + /** True when the connector accepted/delivered the message. */ + delivered: boolean; + /** Connector-assigned message id when available. */ + messageId?: string; + /** Reason when `delivered` is false. */ + error?: string; +} + +/** Liveness of a connector — the "health" half of ack/health. */ +export interface ConnectorHealth { + /** The transport endpoint is reachable. */ + reachable: boolean; + /** Credentials are valid / the connector is authenticated. */ + authenticated: boolean; + /** ISO-8601 of the last successful interaction, if any. */ + lastSeen?: string; + /** Human-readable detail (e.g. failure reason). */ + detail?: string; +} + +/** Unsubscribe handle returned by `subscribe`. */ +export type Unsubscribe = () => void; + +/** + * The uniform contract every orchestrator chat connector implements. Small by + * design — send / subscribe / health — so connectors are interchangeable and + * fleet core never branches on connector kind. + */ +export interface OrchestratorConnector { + /** Which kind of connector this is. */ + readonly kind: ConnectorKind; + /** Send a message from the orchestrator to the operator. */ + send(message: OutboundMessage): Promise; + /** Subscribe to inbound operator messages; returns an unsubscribe handle. */ + subscribe(handler: (message: InboundMessage) => void): Unsubscribe; + /** Report connector liveness (reachable + authenticated). */ + health(): Promise; +} + +/** + * Connector configuration carried by the roster (the `connector` block). + * Secrets (access tokens, bot tokens) are NEVER stored here — they come from + * the environment (the gateway env-config pattern). Absent config ⇒ tmux. + */ +export interface ConnectorConfig { + kind: ConnectorKind; + /** Matrix connector settings (homeserver + room); token via env. */ + matrix?: MatrixConnectorConfig; + /** Discord connector settings (channel); token via env. */ + discord?: DiscordConnectorConfig; +} + +export interface MatrixConnectorConfig { + /** Local homeserver base URL, e.g. https://matrix.example.internal */ + homeserverUrl: string; + /** Full Matrix user id of the orchestrator, e.g. @mos:example.internal */ + userId: string; + /** Room id/alias the orchestrator converses in. */ + roomId: string; +} + +export interface DiscordConnectorConfig { + /** Channel id the orchestrator converses in. */ + channelId: string; +} + +/** The default connector when a roster declares none (back-compat). */ +export const DEFAULT_CONNECTOR_KIND: ConnectorKind = 'tmux';