stack/docs/guides/deployment.md

Deployment Guide

This guide covers deploying Mosaic in two modes: Docker Compose (recommended for quick setup) and bare-metal (production, full control).

---

Prerequisites

Dependency	Minimum version	Notes
Node.js	22 LTS	Required for ESM + --experimental-vm-modules
pnpm	9	npm install -g pnpm
PostgreSQL	17	Must have the pgvector extension
Valkey	8	Redis-compatible; Redis 7+ also works
Docker + Compose	v2	For the Docker Compose path only

---

Docker Compose Deployment (Quick Start)

The docker-compose.yml at the repository root starts PostgreSQL 17 (with pgvector), Valkey 8, an OpenTelemetry Collector, and Jaeger.

1. Clone and configure

git clone <repo-url> mosaic
cd mosaic
cp .env.example .env

Edit .env. The minimum required change is:

BETTER_AUTH_SECRET=<output of: openssl rand -base64 32>

2. Start infrastructure services

docker compose up -d

Services and their ports:

Service	Default port
PostgreSQL	localhost:5433
Valkey	localhost:6380
OTEL Collector (HTTP)	localhost:4318
OTEL Collector (gRPC)	localhost:4317
Jaeger UI	http://localhost:16686

Override host ports via PG_HOST_PORT and VALKEY_HOST_PORT in .env if the defaults conflict.

3. Install dependencies

pnpm install

4. Initialize the database

mosaic-db-migrator --run
mosaic-db-migrator --verify

This is the sole PostgreSQL DDL/readiness control plane. It receives migration-only credentials through deployment secret mounts and accepts no URL, SQL, schema, or role argv.

5. Build all packages

pnpm build

6. Start the gateway

pnpm --filter @mosaicstack/gateway dev

Or for production (after build):

node apps/gateway/dist/main.js

7. Start the web app

# Development
pnpm --filter @mosaicstack/web dev

# Production (after build)
pnpm --filter @mosaicstack/web start

The web app runs on port 3000 by default.

---

Bare-Metal Deployment

Use this path when you want to manage PostgreSQL and Valkey yourself (e.g., existing infrastructure, managed cloud databases).

Step 1 — Install system dependencies

# Node.js 22 via nvm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
nvm install 22
nvm use 22

# pnpm
npm install -g pnpm

# PostgreSQL 17 with pgvector (Debian/Ubuntu example)
sudo apt-get install -y postgresql-17 postgresql-17-pgvector

# Valkey
# Follow https://valkey.io/download/ for your distribution

Step 2 — Provision the database

Do not run ad hoc SQL, create application roles, or create extensions from this guide. The external reviewed bootstrap artifact provisions roles/pgvector, then the migration Job runs mosaic-db-migrator --run and mosaic-db-migrator --verify over verified TLS. This guide does not authorize a plaintext, owner-runtime, or manual-extension bootstrap.

Step 3 — Clone and configure

git clone <repo-url> /opt/mosaic
cd /opt/mosaic

Production deployment does not read a committed or local .env file. Use the reviewed secret renderer/deployment control plane: it injects runtime secrets only at process exec, migration secrets only to the one-shot runner, and non-secret endpoint configuration separately. Validate required values at startup; never write a DSN or BetterAuth secret into this guide or a production filesystem .env.

Step 4 — Install dependencies and build

pnpm install
pnpm build

Step 5 — Run database migrations

mosaic-db-migrator --run
mosaic-db-migrator --verify

The one-shot migrator alone receives the migration secret; Gateway/runtime never receives it.

Step 6 — Start the gateway

node apps/gateway/dist/main.js

The gateway reads .env from the monorepo root automatically (via dotenv in main.ts).

Step 7 — Start the web app

# Next.js standalone output
node apps/web/.next/standalone/server.js

The standalone build is self-contained; it does not require node_modules to be present at runtime.

Step 8 — Configure a reverse proxy

Nginx example

# /etc/nginx/sites-available/mosaic

# Gateway API
server {
    listen 443 ssl;
    server_name your-domain.example.com;

    ssl_certificate     /etc/ssl/certs/your-domain.crt;
    ssl_certificate_key /etc/ssl/private/your-domain.key;

    # WebSocket support (for chat.gateway.ts / Socket.IO)
    location /socket.io/ {
        proxy_pass http://127.0.0.1:14242;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    # REST + auth
    location / {
        proxy_pass http://127.0.0.1:14242;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

# Web app (optional — serve on a subdomain or a separate server block)
server {
    listen 443 ssl;
    server_name app.your-domain.example.com;

    ssl_certificate     /etc/ssl/certs/your-domain.crt;
    ssl_certificate_key /etc/ssl/private/your-domain.key;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Caddy example

# /etc/caddy/Caddyfile

your-domain.example.com {
    reverse_proxy /socket.io/* localhost:14242 {
        header_up Upgrade {http.upgrade}
        header_up Connection {http.connection}
    }
    reverse_proxy localhost:14242
}

app.your-domain.example.com {
    reverse_proxy localhost:3000
}

---

Production Considerations

systemd Services

Create a service unit for each process.

Gateway — /etc/systemd/system/mosaic-gateway.service:

[Unit]
Description=Mosaic Gateway
After=network.target postgresql.service

[Service]
Type=simple
User=mosaic
WorkingDirectory=/opt/mosaic
EnvironmentFile=/opt/mosaic/.env
ExecStart=/usr/bin/node apps/gateway/dist/main.js
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Web app — /etc/systemd/system/mosaic-web.service:

[Unit]
Description=Mosaic Web App
After=network.target mosaic-gateway.service

[Service]
Type=simple
User=mosaic
WorkingDirectory=/opt/mosaic/apps/web
EnvironmentFile=/opt/mosaic/.env
ExecStart=/usr/bin/node .next/standalone/server.js
Environment=PORT=3000
Environment=HOSTNAME=127.0.0.1
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable --now mosaic-gateway mosaic-web

Log Management

Gateway and web app logs go to systemd journal by default. View with:

journalctl -u mosaic-gateway -f
journalctl -u mosaic-web -f

Rotate logs by configuring journald in /etc/systemd/journald.conf:

SystemMaxUse=500M
MaxRetentionSec=30day

Security Checklist

• Set BETTER_AUTH_SECRET to a cryptographically random value (openssl rand -base64 32).
• Restrict GATEWAY_CORS_ORIGIN to your exact frontend origin — do not use *.
• Run services as a dedicated non-root system user (e.g., mosaic).
• Firewall: only expose ports 80/443 externally; keep 14242 and 3000 bound to 127.0.0.1.
• Set AGENT_FILE_SANDBOX_DIR to a directory outside the application root to prevent agent tools from accessing source code.
• If using AGENT_USER_TOOLS, enumerate only the tools non-admin users need.

---

Troubleshooting

Gateway fails to start — "BETTER_AUTH_SECRET is required"

BETTER_AUTH_SECRET is missing or empty. Set it in .env and restart.

Runtime database connection refused

Verify PostgreSQL service/TLS readiness and the deployment-mounted runtime secret reference. Do not print or test a DSN from a shell. Run the sanitized mosaic-db-migrator --verify readiness procedure; if pgvector or schema verification fails, stop and remediate through the reviewed bootstrap/runner control plane. Manual extension SQL is unavailable.

Valkey / Redis connection refused

Check the URL in VALKEY_URL. The Docker Compose default is port 6380.

redis-cli -u "$VALKEY_URL" ping

WebSocket connections fail in production

Ensure your reverse proxy forwards the Upgrade and Connection headers. See the Nginx/Caddy examples above.

Ollama models not appearing

Set OLLAMA_BASE_URL to the URL where Ollama is running (e.g., http://localhost:11434) and set OLLAMA_MODELS to a comma-separated list of model IDs you have pulled.

ollama pull llama3.2

OTEL traces not appearing in Jaeger

Verify the collector is reachable at OTEL_EXPORTER_OTLP_ENDPOINT. With Docker Compose the default is http://localhost:4318. Check docker compose ps and docker compose logs otel-collector.

Summarization / embedding features not working

These features require OPENAI_API_KEY to be set, or you must point SUMMARIZATION_API_URL / EMBEDDING_API_URL to an OpenAI-compatible endpoint (e.g., a local Ollama instance with an embeddings model).