stack/docs/guides/deployment.md

# Deployment Guide

This guide covers deploying Mosaic in two modes: **Docker Compose** (recommended for quick setup) and **bare-metal** (production, full control).

---

## Prerequisites

| Dependency       | Minimum version | Notes                                          |
| ---------------- | --------------- | ---------------------------------------------- |
| Node.js          | 22 LTS          | Required for ESM + `--experimental-vm-modules` |
| pnpm             | 9               | `npm install -g pnpm`                          |
| PostgreSQL       | 17              | Must have the `pgvector` extension             |
| Valkey           | 8               | Redis-compatible; Redis 7+ also works          |
| Docker + Compose | v2              | For the Docker Compose path only               |

---

## Docker Compose Deployment (Quick Start)

The `docker-compose.yml` at the repository root starts PostgreSQL 17 (with pgvector), Valkey 8, an OpenTelemetry Collector, and Jaeger.

### 1. Clone and configure

```bash
git clone <repo-url> mosaic
cd mosaic
cp .env.example .env
```

Edit `.env`. The minimum required change is:

```dotenv
BETTER_AUTH_SECRET=<output of: openssl rand -base64 32>
```

### 2. Start infrastructure services

```bash
docker compose up -d
```

Services and their ports:

| Service               | Default port             |
| --------------------- | ------------------------ |
| PostgreSQL            | `localhost:5433`         |
| Valkey                | `localhost:6380`         |
| OTEL Collector (HTTP) | `localhost:4318`         |
| OTEL Collector (gRPC) | `localhost:4317`         |
| Jaeger UI             | `http://localhost:16686` |

Override host ports via `PG_HOST_PORT` and `VALKEY_HOST_PORT` in `.env` if the defaults conflict.

### 3. Install dependencies

```bash
pnpm install
```

### 4. Initialize the database

```bash
pnpm --filter @mosaicstack/db db:migrate
```

### 5. Build all packages

```bash
pnpm build
```

### 6. Start the gateway

```bash
pnpm --filter @mosaicstack/gateway dev
```

Or for production (after build):

```bash
node apps/gateway/dist/main.js
```

### 7. Start the web app

```bash
# Development
pnpm --filter @mosaicstack/web dev

# Production (after build)
pnpm --filter @mosaicstack/web start
```

The web app runs on port `3000` by default.

---

## Bare-Metal Deployment

Use this path when you want to manage PostgreSQL and Valkey yourself (e.g., existing infrastructure, managed cloud databases).

### Step 1 — Install system dependencies

```bash
# Node.js 22 via nvm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
nvm install 22
nvm use 22

# pnpm
npm install -g pnpm

# PostgreSQL 17 with pgvector (Debian/Ubuntu example)
sudo apt-get install -y postgresql-17 postgresql-17-pgvector

# Valkey
# Follow https://valkey.io/download/ for your distribution
```

### Step 2 — Create the database

```sql
-- Run as the postgres superuser
CREATE USER mosaic WITH PASSWORD 'change-me';
CREATE DATABASE mosaic OWNER mosaic;
\c mosaic
CREATE EXTENSION IF NOT EXISTS vector;
```

### Step 3 — Clone and configure

```bash
git clone <repo-url> /opt/mosaic
cd /opt/mosaic
cp .env.example .env
```

Edit `/opt/mosaic/.env`. Required fields:

```dotenv
DATABASE_URL=postgresql://mosaic:<password>@localhost:5432/mosaic
VALKEY_URL=redis://localhost:6379
BETTER_AUTH_SECRET=<openssl rand -base64 32>
BETTER_AUTH_URL=https://your-domain.example.com
GATEWAY_CORS_ORIGIN=https://your-domain.example.com
NEXT_PUBLIC_GATEWAY_URL=https://your-domain.example.com
```

### Step 4 — Install dependencies and build

```bash
pnpm install
pnpm build
```

### Step 5 — Run database migrations

```bash
pnpm --filter @mosaicstack/db db:migrate
```

### Step 6 — Start the gateway

```bash
node apps/gateway/dist/main.js
```

The gateway reads `.env` from the monorepo root automatically (via `dotenv` in `main.ts`).

### Step 7 — Start the web app

```bash
# Next.js standalone output
node apps/web/.next/standalone/server.js
```

The standalone build is self-contained; it does not require `node_modules` to be present at runtime.

### Step 8 — Configure a reverse proxy

#### Nginx example

```nginx
# /etc/nginx/sites-available/mosaic

# Gateway API
server {
    listen 443 ssl;
    server_name your-domain.example.com;

    ssl_certificate     /etc/ssl/certs/your-domain.crt;
    ssl_certificate_key /etc/ssl/private/your-domain.key;

    # WebSocket support (for chat.gateway.ts / Socket.IO)
    location /socket.io/ {
        proxy_pass http://127.0.0.1:14242;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    # REST + auth
    location / {
        proxy_pass http://127.0.0.1:14242;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

# Web app (optional — serve on a subdomain or a separate server block)
server {
    listen 443 ssl;
    server_name app.your-domain.example.com;

    ssl_certificate     /etc/ssl/certs/your-domain.crt;
    ssl_certificate_key /etc/ssl/private/your-domain.key;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}
```

#### Caddy example

```caddyfile
# /etc/caddy/Caddyfile

your-domain.example.com {
    reverse_proxy /socket.io/* localhost:14242 {
        header_up Upgrade {http.upgrade}
        header_up Connection {http.connection}
    }
    reverse_proxy localhost:14242
}

app.your-domain.example.com {
    reverse_proxy localhost:3000
}
```

---

## Production Considerations

### systemd Services

Create a service unit for each process.

**Gateway** — `/etc/systemd/system/mosaic-gateway.service`:

```ini
[Unit]
Description=Mosaic Gateway
After=network.target postgresql.service

[Service]
Type=simple
User=mosaic
WorkingDirectory=/opt/mosaic
EnvironmentFile=/opt/mosaic/.env
ExecStart=/usr/bin/node apps/gateway/dist/main.js
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
```

**Web app** — `/etc/systemd/system/mosaic-web.service`:

```ini
[Unit]
Description=Mosaic Web App
After=network.target mosaic-gateway.service

[Service]
Type=simple
User=mosaic
WorkingDirectory=/opt/mosaic/apps/web
EnvironmentFile=/opt/mosaic/.env
ExecStart=/usr/bin/node .next/standalone/server.js
Environment=PORT=3000
Environment=HOSTNAME=127.0.0.1
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
```

Enable and start:

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now mosaic-gateway mosaic-web
```

### Log Management

Gateway and web app logs go to systemd journal by default. View with:

```bash
journalctl -u mosaic-gateway -f
journalctl -u mosaic-web -f
```

Rotate logs by configuring `journald` in `/etc/systemd/journald.conf`:

```ini
SystemMaxUse=500M
MaxRetentionSec=30day
```

### Security Checklist

- Set `BETTER_AUTH_SECRET` to a cryptographically random value (`openssl rand -base64 32`).
- Restrict `GATEWAY_CORS_ORIGIN` to your exact frontend origin — do not use `*`.
- Run services as a dedicated non-root system user (e.g., `mosaic`).
- Firewall: only expose ports 80/443 externally; keep 14242 and 3000 bound to `127.0.0.1`.
- Set `AGENT_FILE_SANDBOX_DIR` to a directory outside the application root to prevent agent tools from accessing source code.
- If using `AGENT_USER_TOOLS`, enumerate only the tools non-admin users need.

---

## Troubleshooting

### Gateway fails to start — "BETTER_AUTH_SECRET is required"

`BETTER_AUTH_SECRET` is missing or empty. Set it in `.env` and restart.

### `DATABASE_URL` connection refused

Verify PostgreSQL is running and the port matches. The Docker Compose default is `5433`; bare-metal typically uses `5432`.

```bash
psql "$DATABASE_URL" -c '\conninfo'
```

### pgvector extension missing

```sql
\c mosaic
CREATE EXTENSION IF NOT EXISTS vector;
```

### Valkey / Redis connection refused

Check the URL in `VALKEY_URL`. The Docker Compose default is port `6380`.

```bash
redis-cli -u "$VALKEY_URL" ping
```

### WebSocket connections fail in production

Ensure your reverse proxy forwards the `Upgrade` and `Connection` headers. See the Nginx/Caddy examples above.

### Ollama models not appearing

Set `OLLAMA_BASE_URL` to the URL where Ollama is running (e.g., `http://localhost:11434`) and set `OLLAMA_MODELS` to a comma-separated list of model IDs you have pulled.

```bash
ollama pull llama3.2
```

### OTEL traces not appearing in Jaeger

Verify the collector is reachable at `OTEL_EXPORTER_OTLP_ENDPOINT`. With Docker Compose the default is `http://localhost:4318`. Check `docker compose ps` and `docker compose logs otel-collector`.

### Summarization / embedding features not working

These features require `OPENAI_API_KEY` to be set, or you must point `SUMMARIZATION_API_URL` / `EMBEDDING_API_URL` to an OpenAI-compatible endpoint (e.g., a local Ollama instance with an embeddings model).