stack/docs/MATRIX-BRIDGE.md

# Matrix Bridge

Integration between Mosaic Stack and the Matrix protocol, enabling workspace management
and job orchestration through Matrix chat rooms.

## Overview

The Matrix bridge connects Mosaic Stack to any Matrix homeserver (Synapse, Dendrite, Conduit,
etc.), allowing users to interact with the platform through Matrix clients like Element,
FluffyChat, or any other Matrix-compatible application.

Key capabilities:

- **Command interface** -- Issue bot commands (`@mosaic fix #42`) from any mapped Matrix room
- **Workspace-room mapping** -- Each Mosaic workspace can be linked to a Matrix room
- **Threaded job updates** -- Job progress is posted to MSC3440 threads, keeping rooms clean
- **Streaming AI responses** -- LLM output streams to Matrix via rate-limited message edits
- **Multi-provider broadcasting** -- HeraldService broadcasts status updates to all active
  chat providers (Discord and Matrix can run simultaneously)

### Architecture

```
Matrix Client (Element, FluffyChat, etc.)
         |
         v
  Synapse Homeserver
         |
    matrix-bot-sdk
         |
         v
+------------------+       +---------------------+
|  MatrixService   |<----->| CommandParserService |
|  (IChatProvider) |       | (shared, all platforms)
+------------------+       +---------------------+
    |         |
    |         v
    |   +--------------------+
    |   | MatrixRoomService  |  workspace <-> room mapping
    |   +--------------------+
    |         |
    v         v
+------------------+     +----------------+
| StitcherService  |     | PrismaService  |
| (job dispatch)   |     | (database)     |
+------------------+     +----------------+
         |
         v
+------------------+
|  HeraldService   |  broadcasts to CHAT_PROVIDERS[]
+------------------+
         |
         v
+---------------------------+
| MatrixStreamingService    |  streaming AI responses
| (m.replace edits, typing) |
+---------------------------+
```

## Quick Start

### 1. Start the dev environment

The Matrix dev environment uses a Docker Compose overlay that adds Synapse and Element Web
alongside the existing Mosaic Stack services.

```bash
# Using Makefile (recommended)
make matrix-up

# Or manually
docker compose -f docker/docker-compose.yml -f docker/docker-compose.matrix.yml up -d
```

This starts:

| Service     | URL                   | Purpose                 |
| ----------- | --------------------- | ----------------------- |
| Synapse     | http://localhost:8008 | Matrix homeserver       |
| Element Web | http://localhost:8501 | Web-based Matrix client |

Both services share the existing Mosaic PostgreSQL instance. A `synapse-db-init` container
runs once to create the `synapse` database and user, then exits.

### 2. Create the bot account

After Synapse is healthy, run the setup script to create admin and bot accounts:

```bash
make matrix-setup-bot

# Or directly
docker/matrix/scripts/setup-bot.sh
```

The script:

1. Registers an admin account (`admin` / `admin-dev-password`)
2. Obtains an admin access token
3. Creates the bot account (`mosaic-bot` / `mosaic-bot-dev-password`)
4. Retrieves the bot access token
5. Prints the environment variables to add to `.env`

Custom credentials can be passed:

```bash
docker/matrix/scripts/setup-bot.sh \
  --username custom-bot \
  --password custom-pass \
  --admin-username myadmin \
  --admin-password myadmin-pass
```

### 3. Configure environment variables

Copy the output from the setup script into your `.env` file:

```bash
# Matrix Bridge Configuration
MATRIX_HOMESERVER_URL=http://localhost:8008
MATRIX_ACCESS_TOKEN=<token from setup-bot.sh>
MATRIX_BOT_USER_ID=@mosaic-bot:localhost
MATRIX_CONTROL_ROOM_ID=!roomid:localhost
MATRIX_WORKSPACE_ID=<your-workspace-uuid>
```

If running the API inside the Docker Compose network, use the internal hostname:

```bash
MATRIX_HOMESERVER_URL=http://synapse:8008
```

### 4. Restart the API

```bash
pnpm dev:api
# or
make docker-restart
```

The BridgeModule will detect `MATRIX_ACCESS_TOKEN` and enable the Matrix bridge
automatically.

### 5. Test in Element Web

1. Open http://localhost:8501
2. Register or log in with any account
3. Create a room and invite `@mosaic-bot:localhost`
4. Send `@mosaic help` or `!mosaic help`

## Configuration

### Environment Variables

| Variable                 | Description                                   | Example                       |
| ------------------------ | --------------------------------------------- | ----------------------------- |
| `MATRIX_HOMESERVER_URL`  | Matrix server URL                             | `http://localhost:8008`       |
| `MATRIX_ACCESS_TOKEN`    | Bot access token (from setup script or login) | `syt_bW9z...`                 |
| `MATRIX_BOT_USER_ID`     | Bot's full Matrix user ID                     | `@mosaic-bot:localhost`       |
| `MATRIX_CONTROL_ROOM_ID` | Default room for status broadcasts            | `!abcdef:localhost`           |
| `MATRIX_WORKSPACE_ID`    | Default workspace UUID for the control room   | `550e8400-e29b-41d4-a716-...` |

All variables are read from `process.env` at service construction time. The bridge activates
only when `MATRIX_ACCESS_TOKEN` is set.

### Dev Environment Variables (docker-compose.matrix.yml)

These configure the local Synapse and Element Web instances:

| Variable                    | Default                | Purpose                   |
| --------------------------- | ---------------------- | ------------------------- |
| `SYNAPSE_POSTGRES_DB`       | `synapse`              | Synapse database name     |
| `SYNAPSE_POSTGRES_USER`     | `synapse`              | Synapse database user     |
| `SYNAPSE_POSTGRES_PASSWORD` | `synapse_dev_password` | Synapse database password |
| `SYNAPSE_CLIENT_PORT`       | `8008`                 | Synapse client API port   |
| `SYNAPSE_FEDERATION_PORT`   | `8448`                 | Synapse federation port   |
| `ELEMENT_PORT`              | `8501`                 | Element Web port          |

## Architecture

### Service Responsibilities

**MatrixService** (`apps/api/src/bridge/matrix/matrix.service.ts`)

The primary Matrix integration. Implements the `IChatProvider` interface.

- Connects to the homeserver using `matrix-bot-sdk`
- Listens for `room.message` events in all joined rooms
- Resolves workspace context via MatrixRoomService (or falls back to control room)
- Normalizes `!mosaic` prefix to `@mosaic` for the shared CommandParserService
- Dispatches parsed commands to StitcherService for job execution
- Creates MSC3440 threads for job updates
- Auto-joins rooms when invited (`AutojoinRoomsMixin`)

**MatrixRoomService** (`apps/api/src/bridge/matrix/matrix-room.service.ts`)

Manages the mapping between Mosaic workspaces and Matrix rooms.

- **Provision**: Creates a private Matrix room named `Mosaic: {workspace_name}` with alias
  `#mosaic-{slug}:{server}`
- **Link/Unlink**: Maps existing rooms to workspaces via `workspace.matrixRoomId`
- **Lookup**: Forward lookup (workspace -> room) and reverse lookup (room -> workspace)
- Room mappings are stored in the `workspace` table's `matrixRoomId` column

**MatrixStreamingService** (`apps/api/src/bridge/matrix/matrix-streaming.service.ts`)

Streams AI responses to Matrix rooms using incremental message edits.

- Sends an initial "Thinking..." placeholder message
- Activates typing indicator during generation
- Buffers incoming tokens and edits the message every 500ms (rate-limited)
- On completion, sends a final clean edit with optional token usage stats
- On error, edits the message with an error notice
- Supports threaded responses via MSC3440

**CommandParserService** (`apps/api/src/bridge/parser/command-parser.service.ts`)

Shared, platform-agnostic command parser used by both Discord and Matrix bridges.

- Parses `@mosaic <action> [args]` commands
- Supports issue references in multiple formats: `#42`, `owner/repo#42`, full URL
- Returns typed `ParsedCommand` objects or structured parse errors with help text

**BridgeModule** (`apps/api/src/bridge/bridge.module.ts`)

Conditional module loader. Inspects environment variables at startup:

- If `DISCORD_BOT_TOKEN` is set, Discord bridge is added to `CHAT_PROVIDERS`
- If `MATRIX_ACCESS_TOKEN` is set, Matrix bridge is added to `CHAT_PROVIDERS`
- Both can run simultaneously; neither is a dependency of the other

**HeraldService** (`apps/api/src/herald/herald.service.ts`)

Status broadcaster that sends job event updates to all active chat providers.

- Iterates over the `CHAT_PROVIDERS` injection token
- Sends thread messages for job lifecycle events (created, started, completed, failed, etc.)
- Uses PDA-friendly language (no "OVERDUE", "URGENT", etc.)
- If one provider fails, others still receive the broadcast

### Data Flow

```
1. User sends "@mosaic fix #42" in a Matrix room
2. MatrixService receives room.message event
3. MatrixRoomService resolves room -> workspace mapping
4. CommandParserService parses the command (action=FIX, issue=#42)
5. MatrixService creates a thread (MSC3440) for job updates
6. StitcherService dispatches the job with workspace context
7. HeraldService receives job events and broadcasts to all CHAT_PROVIDERS
8. Thread messages appear in the Matrix room thread
```

### Thread Model (MSC3440)

Matrix threads are implemented per [MSC3440](https://github.com/matrix-org/matrix-spec-proposals/pull/3440):

- A **thread root** is created by sending a regular `m.room.message` event
- Subsequent messages reference the root via `m.relates_to` with `rel_type: "m.thread"`
- The `is_falling_back: true` flag and `m.in_reply_to` provide compatibility with clients
  that do not support threads
- Thread root event IDs are stored in job metadata for HeraldService to post updates

## Commands

All commands accept either `@mosaic` or `!mosaic` prefix. The `!mosaic` form is
normalized to `@mosaic` internally before parsing.

| Command                    | Description                   | Example                      |
| -------------------------- | ----------------------------- | ---------------------------- |
| `@mosaic fix <issue>`      | Start a job for an issue      | `@mosaic fix #42`            |
| `@mosaic status <job-id>`  | Check job status              | `@mosaic status job-abc123`  |
| `@mosaic cancel <job-id>`  | Cancel a running job          | `@mosaic cancel job-abc123`  |
| `@mosaic retry <job-id>`   | Retry a failed job            | `@mosaic retry job-abc123`   |
| `@mosaic verbose <job-id>` | Stream full logs to thread    | `@mosaic verbose job-abc123` |
| `@mosaic quiet`            | Reduce notification verbosity | `@mosaic quiet`              |
| `@mosaic help`             | Show available commands       | `@mosaic help`               |

### Issue Reference Formats

The `fix` command accepts issue references in multiple formats:

```
@mosaic fix #42                                    # Current repo
@mosaic fix owner/repo#42                          # Cross-repo
@mosaic fix https://git.example.com/o/r/issues/42  # Full URL
```

### Noise Management

Job updates are scoped to threads to keep main rooms clean:

- **Main room**: Low verbosity -- milestone completions only
- **Job threads**: Medium verbosity -- step completions and status changes
- **DMs**: Configurable per user (planned)

## Workspace-Room Mapping

Each Mosaic workspace can be associated with one Matrix room. The mapping is stored in the
`workspace` table's `matrixRoomId` column.

### Automatic Provisioning

When a workspace needs a Matrix room, MatrixRoomService provisions one:

```
Room name:  "Mosaic: My Workspace"
Room alias: #mosaic-my-workspace:localhost
Visibility: private
```

The room ID is then stored in `workspace.matrixRoomId`.

### Manual Linking

Existing rooms can be linked to workspaces:

```typescript
await matrixRoomService.linkWorkspaceToRoom(workspaceId, "!roomid:localhost");
```

And unlinked:

```typescript
await matrixRoomService.unlinkWorkspace(workspaceId);
```

### Message Routing

When a message arrives in a room:

1. MatrixRoomService performs a reverse lookup: room ID -> workspace ID
2. If no mapping is found, the service checks if the room is the configured control room
   (`MATRIX_CONTROL_ROOM_ID`) and uses `MATRIX_WORKSPACE_ID` as fallback
3. If still unmapped, the message is ignored

This ensures commands only execute within a valid workspace context.

## Streaming Responses

MatrixStreamingService enables real-time AI response streaming in Matrix rooms.

### How It Works

1. An initial placeholder message ("Thinking...") is sent to the room
2. The bot's typing indicator is activated
3. Tokens from the LLM arrive via an `AsyncIterable<string>`
4. Tokens are buffered and the message is edited via `m.replace` events
5. Edits are rate-limited to a maximum of once every **500ms** to avoid flooding the
   homeserver
6. When streaming completes, a final clean edit is sent and the typing indicator clears
7. On error, the message is edited to include an error notice

### Message Edit Format (m.replace)

```json
{
  "m.new_content": {
    "msgtype": "m.text",
    "body": "Updated response text"
  },
  "m.relates_to": {
    "rel_type": "m.replace",
    "event_id": "$original_event_id"
  },
  "msgtype": "m.text",
  "body": "* Updated response text"
}
```

The top-level `body` prefixed with `*` serves as a fallback for clients that do not
support message edits.

### Thread Support

Streaming responses can target a specific thread by passing `threadId` in the options.
The initial message and all edits will include the `m.thread` relation.

## Development

### Running Tests

```bash
# All bridge tests
pnpm test -- --filter @mosaic/api -- matrix

# Individual service tests
pnpm test -- --filter @mosaic/api -- matrix.service
pnpm test -- --filter @mosaic/api -- matrix-room.service
pnpm test -- --filter @mosaic/api -- matrix-streaming.service
pnpm test -- --filter @mosaic/api -- command-parser
pnpm test -- --filter @mosaic/api -- bridge.module
```

### Adding a New Command

1. Add the action to the `CommandAction` enum in
   `apps/api/src/bridge/parser/command.interface.ts`

2. Add parsing logic in `CommandParserService.parseActionArguments()`
   (`apps/api/src/bridge/parser/command-parser.service.ts`)

3. Add the handler case in `MatrixService.handleParsedCommand()`
   (`apps/api/src/bridge/matrix/matrix.service.ts`)

4. Implement the handler method (e.g., `handleNewCommand()`)

5. Update the help text in `MatrixService.handleHelpCommand()`

6. Add tests for the new command in both the parser and service spec files

### Extending the Bridge

The `IChatProvider` interface (`apps/api/src/bridge/interfaces/chat-provider.interface.ts`)
defines the contract all chat bridges implement:

```typescript
interface IChatProvider {
  connect(): Promise<void>;
  disconnect(): Promise<void>;
  isConnected(): boolean;
  sendMessage(channelId: string, content: string): Promise<void>;
  createThread(options: ThreadCreateOptions): Promise<string>;
  sendThreadMessage(options: ThreadMessageOptions): Promise<void>;
  parseCommand(message: ChatMessage): ChatCommand | null;
  editMessage?(channelId: string, messageId: string, content: string): Promise<void>;
}
```

To add a new chat platform:

1. Create a new service implementing `IChatProvider`
2. Register it in `BridgeModule` with a conditional check on its environment variable
3. Add it to the `CHAT_PROVIDERS` factory
4. HeraldService will automatically broadcast to it with no further changes

### File Layout

```
apps/api/src/
  bridge/
    bridge.module.ts              # Conditional module loader
    bridge.constants.ts           # CHAT_PROVIDERS injection token
    interfaces/
      chat-provider.interface.ts  # IChatProvider contract
      index.ts
    parser/
      command-parser.service.ts   # Shared command parser
      command-parser.spec.ts
      command.interface.ts        # Command types and enums
    matrix/
      matrix.service.ts           # Core Matrix integration
      matrix.service.spec.ts
      matrix-room.service.ts      # Workspace-room mapping
      matrix-room.service.spec.ts
      matrix-streaming.service.ts # Streaming AI responses
      matrix-streaming.service.spec.ts
    discord/
      discord.service.ts          # Discord integration (parallel)
  herald/
    herald.module.ts
    herald.service.ts             # Status broadcasting
    herald.service.spec.ts

docker/
  docker-compose.matrix.yml        # Dev overlay (Synapse + Element)
  docker-compose.sample.matrix.yml # Production sample (Swarm)
  matrix/
    synapse/
      homeserver.yaml             # Dev Synapse config
    element/
      config.json                 # Dev Element Web config
    scripts/
      setup-bot.sh                # Bot account setup
```

## Deployment

### Production Considerations

The dev environment uses relaxed settings that are not suitable for production.
Review and address the following before deploying:

**Synapse Configuration**

- Set a proper `server_name` (this is permanent and cannot change after first run)
- Disable open registration (`enable_registration: false`)
- Replace dev secrets (`macaroon_secret_key`, `form_secret`) with strong random values
- Configure proper rate limiting (dev config allows 100 msg/sec)
- Set up TLS termination (via reverse proxy or Synapse directly)
- Consider a dedicated PostgreSQL instance rather than the shared Mosaic database

**Bot Security**

- Generate a strong bot password (not the dev default)
- Store the access token securely (use a secrets manager or encrypted `.env`)
- The bot auto-joins rooms when invited -- consider restricting this in production
  by removing `AutojoinRoomsMixin` and implementing allow-list logic

**Environment Variables**

- `MATRIX_WORKSPACE_ID` should be a valid workspace UUID from your database; all
  commands from the control room execute within this workspace context

**Network**

- If Synapse runs on a separate host, ensure `MATRIX_HOMESERVER_URL` points to the
  correct endpoint
- For federation, configure DNS SRV records and `.well-known` delegation

### Sample Production Stack

A production-ready Docker Swarm compose file is provided at
`docker/docker-compose.sample.matrix.yml`. It includes:

- Synapse with Traefik labels for automatic TLS
- Element Web with its own domain
- Dedicated PostgreSQL instance for Synapse
- Optional coturn (TURN/STUN) for voice/video

Deploy via Portainer or Docker Swarm CLI:

```bash
docker stack deploy -c docker/docker-compose.sample.matrix.yml matrix
```

After deploying, follow the post-deploy steps in the compose file comments to create
accounts and configure the Mosaic Stack connection.

### Makefile Targets

| Target                  | Description                               |
| ----------------------- | ----------------------------------------- |
| `make matrix-up`        | Start Synapse + Element Web (dev overlay) |
| `make matrix-down`      | Stop Matrix services                      |
| `make matrix-logs`      | Follow Synapse and Element logs           |
| `make matrix-setup-bot` | Run bot account setup script              |