Compare commits
4 Commits
fa717999d9
...
9f44a390ba
| Author | SHA1 | Date | |
|---|---|---|---|
| 9f44a390ba | |||
| 95e8c0ce2b | |||
| a7804e689d | |||
| 75a844ca92 |
124
.env.example
124
.env.example
@@ -1,129 +1,35 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Mosaic — Environment Variables Reference
|
||||
# Copy this file to .env and fill in the values for your deployment.
|
||||
# Lines beginning with # are comments; optional vars are commented out.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
# ─── Database (PostgreSQL 17 + pgvector) ─────────────────────────────────────
|
||||
# Full connection string used by the gateway, ORM, and migration runner.
|
||||
# Port 5433 avoids conflict with a host-side PostgreSQL instance.
|
||||
# Database (port 5433 avoids conflict with host PostgreSQL)
|
||||
DATABASE_URL=postgresql://mosaic:mosaic@localhost:5433/mosaic
|
||||
|
||||
# Docker Compose host-port override for the PostgreSQL container (default: 5433)
|
||||
# PG_HOST_PORT=5433
|
||||
|
||||
|
||||
# ─── Queue (Valkey 8 / Redis-compatible) ─────────────────────────────────────
|
||||
# Port 6380 avoids conflict with a host-side Redis/Valkey instance.
|
||||
# Valkey (Redis-compatible, port 6380 avoids conflict with host Redis/Valkey)
|
||||
VALKEY_URL=redis://localhost:6380
|
||||
|
||||
# Docker Compose host-port override for the Valkey container (default: 6380)
|
||||
# Docker Compose host port overrides (optional)
|
||||
# PG_HOST_PORT=5433
|
||||
# VALKEY_HOST_PORT=6380
|
||||
|
||||
|
||||
# ─── Gateway ─────────────────────────────────────────────────────────────────
|
||||
# TCP port the NestJS/Fastify gateway listens on (default: 4000)
|
||||
GATEWAY_PORT=4000
|
||||
|
||||
# Comma-separated list of allowed CORS origins.
|
||||
# Must include the web app origin in production.
|
||||
GATEWAY_CORS_ORIGIN=http://localhost:3000
|
||||
|
||||
|
||||
# ─── Auth (BetterAuth) ───────────────────────────────────────────────────────
|
||||
# REQUIRED — random secret used to sign sessions and tokens.
|
||||
# Generate with: openssl rand -base64 32
|
||||
BETTER_AUTH_SECRET=change-me-to-a-random-32-char-string
|
||||
|
||||
# Public base URL of the gateway (used by BetterAuth for callback URLs)
|
||||
BETTER_AUTH_URL=http://localhost:4000
|
||||
|
||||
|
||||
# ─── Web App (Next.js) ───────────────────────────────────────────────────────
|
||||
# Public gateway URL — accessible from the browser, not just the server.
|
||||
NEXT_PUBLIC_GATEWAY_URL=http://localhost:4000
|
||||
|
||||
|
||||
# ─── OpenTelemetry ───────────────────────────────────────────────────────────
|
||||
# OTLP HTTP endpoint (otel-collector or any OpenTelemetry-compatible backend)
|
||||
# OpenTelemetry
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
|
||||
|
||||
# Service name shown in traces
|
||||
OTEL_SERVICE_NAME=mosaic-gateway
|
||||
|
||||
# Auth (BetterAuth)
|
||||
BETTER_AUTH_SECRET=change-me-to-a-random-32-char-string
|
||||
BETTER_AUTH_URL=http://localhost:4000
|
||||
|
||||
# ─── AI Providers ────────────────────────────────────────────────────────────
|
||||
# Gateway
|
||||
GATEWAY_PORT=4000
|
||||
GATEWAY_CORS_ORIGIN=http://localhost:3000
|
||||
|
||||
# Ollama (local models — set OLLAMA_BASE_URL to enable)
|
||||
# OLLAMA_BASE_URL=http://localhost:11434
|
||||
# OLLAMA_HOST is a legacy alias for OLLAMA_BASE_URL
|
||||
# OLLAMA_HOST=http://localhost:11434
|
||||
# Comma-separated list of Ollama model IDs to register (default: llama3.2,codellama,mistral)
|
||||
# OLLAMA_MODELS=llama3.2,codellama,mistral
|
||||
|
||||
# OpenAI — required for embedding and log-summarization features
|
||||
# OPENAI_API_KEY=sk-...
|
||||
|
||||
# Custom providers — JSON array of provider configs
|
||||
# Format: [{"id":"<id>","baseUrl":"<url>","apiKey":"<key>","models":[{"id":"<model-id>","name":"<label>"}]}]
|
||||
# MOSAIC_CUSTOM_PROVIDERS=
|
||||
|
||||
|
||||
# ─── Embedding Service ───────────────────────────────────────────────────────
|
||||
# OpenAI-compatible embeddings endpoint (default: OpenAI)
|
||||
# EMBEDDING_API_URL=https://api.openai.com/v1
|
||||
# EMBEDDING_MODEL=text-embedding-3-small
|
||||
|
||||
|
||||
# ─── Log Summarization Service ───────────────────────────────────────────────
|
||||
# OpenAI-compatible chat completions endpoint for log summarization (default: OpenAI)
|
||||
# SUMMARIZATION_API_URL=https://api.openai.com/v1
|
||||
# SUMMARIZATION_MODEL=gpt-4o-mini
|
||||
|
||||
# Cron schedule for summarization job (default: every 6 hours)
|
||||
# SUMMARIZATION_CRON=0 */6 * * *
|
||||
|
||||
# Cron schedule for log tier management (default: daily at 03:00)
|
||||
# TIER_MANAGEMENT_CRON=0 3 * * *
|
||||
|
||||
|
||||
# ─── Agent ───────────────────────────────────────────────────────────────────
|
||||
# Filesystem sandbox root for agent file tools (default: process.cwd())
|
||||
# AGENT_FILE_SANDBOX_DIR=/var/lib/mosaic/sandbox
|
||||
|
||||
# Comma-separated list of tool names available to non-admin users.
|
||||
# Leave unset to allow all tools for all authenticated users.
|
||||
# AGENT_USER_TOOLS=read_file,list_directory,search_files
|
||||
|
||||
# System prompt injected into every agent session (optional)
|
||||
# AGENT_SYSTEM_PROMPT=You are a helpful assistant.
|
||||
|
||||
|
||||
# ─── MCP Servers ─────────────────────────────────────────────────────────────
|
||||
# JSON array of MCP server configs — set to enable MCP tool integration.
|
||||
# Each entry: {"name":"<id>","url":"<http-or-sse-url>"}
|
||||
# MCP_SERVERS=[{"name":"my-mcp","url":"http://localhost:3100/sse"}]
|
||||
|
||||
|
||||
# ─── Coordinator ─────────────────────────────────────────────────────────────
|
||||
# Root directory used to scope coordinator (worktree/repo) operations.
|
||||
# Defaults to the monorepo root auto-detected from process.cwd().
|
||||
# MOSAIC_WORKSPACE_ROOT=/home/user/projects/mosaic
|
||||
|
||||
|
||||
# ─── Discord Plugin (optional — set DISCORD_BOT_TOKEN to enable) ─────────────
|
||||
# Discord Plugin (optional — set DISCORD_BOT_TOKEN to enable)
|
||||
# DISCORD_BOT_TOKEN=
|
||||
# DISCORD_GUILD_ID=
|
||||
# DISCORD_GATEWAY_URL=http://localhost:4000
|
||||
|
||||
|
||||
# ─── Telegram Plugin (optional — set TELEGRAM_BOT_TOKEN to enable) ───────────
|
||||
# Telegram Plugin (optional — set TELEGRAM_BOT_TOKEN to enable)
|
||||
# TELEGRAM_BOT_TOKEN=
|
||||
# TELEGRAM_GATEWAY_URL=http://localhost:4000
|
||||
|
||||
|
||||
# ─── Authentik SSO (optional — set AUTHENTIK_CLIENT_ID to enable) ────────────
|
||||
# AUTHENTIK_ISSUER=https://auth.example.com/application/o/mosaic/
|
||||
# Authentik SSO (optional — set AUTHENTIK_CLIENT_ID to enable)
|
||||
# AUTHENTIK_ISSUER=https://auth.example.com
|
||||
# AUTHENTIK_CLIENT_ID=
|
||||
# AUTHENTIK_CLIENT_SECRET=
|
||||
|
||||
@@ -1,384 +0,0 @@
|
||||
# Deployment Guide
|
||||
|
||||
This guide covers deploying Mosaic in two modes: **Docker Compose** (recommended for quick setup) and **bare-metal** (production, full control).
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Dependency | Minimum version | Notes |
|
||||
| ---------------- | --------------- | ---------------------------------------------- |
|
||||
| Node.js | 22 LTS | Required for ESM + `--experimental-vm-modules` |
|
||||
| pnpm | 9 | `npm install -g pnpm` |
|
||||
| PostgreSQL | 17 | Must have the `pgvector` extension |
|
||||
| Valkey | 8 | Redis-compatible; Redis 7+ also works |
|
||||
| Docker + Compose | v2 | For the Docker Compose path only |
|
||||
|
||||
---
|
||||
|
||||
## Docker Compose Deployment (Quick Start)
|
||||
|
||||
The `docker-compose.yml` at the repository root starts PostgreSQL 17 (with pgvector), Valkey 8, an OpenTelemetry Collector, and Jaeger.
|
||||
|
||||
### 1. Clone and configure
|
||||
|
||||
```bash
|
||||
git clone <repo-url> mosaic
|
||||
cd mosaic
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Edit `.env`. The minimum required change is:
|
||||
|
||||
```dotenv
|
||||
BETTER_AUTH_SECRET=<output of: openssl rand -base64 32>
|
||||
```
|
||||
|
||||
### 2. Start infrastructure services
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Services and their ports:
|
||||
|
||||
| Service | Default port |
|
||||
| --------------------- | ------------------------ |
|
||||
| PostgreSQL | `localhost:5433` |
|
||||
| Valkey | `localhost:6380` |
|
||||
| OTEL Collector (HTTP) | `localhost:4318` |
|
||||
| OTEL Collector (gRPC) | `localhost:4317` |
|
||||
| Jaeger UI | `http://localhost:16686` |
|
||||
|
||||
Override host ports via `PG_HOST_PORT` and `VALKEY_HOST_PORT` in `.env` if the defaults conflict.
|
||||
|
||||
### 3. Install dependencies
|
||||
|
||||
```bash
|
||||
pnpm install
|
||||
```
|
||||
|
||||
### 4. Initialize the database
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaic/db db:migrate
|
||||
```
|
||||
|
||||
### 5. Build all packages
|
||||
|
||||
```bash
|
||||
pnpm build
|
||||
```
|
||||
|
||||
### 6. Start the gateway
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaic/gateway dev
|
||||
```
|
||||
|
||||
Or for production (after build):
|
||||
|
||||
```bash
|
||||
node apps/gateway/dist/main.js
|
||||
```
|
||||
|
||||
### 7. Start the web app
|
||||
|
||||
```bash
|
||||
# Development
|
||||
pnpm --filter @mosaic/web dev
|
||||
|
||||
# Production (after build)
|
||||
pnpm --filter @mosaic/web start
|
||||
```
|
||||
|
||||
The web app runs on port `3000` by default.
|
||||
|
||||
---
|
||||
|
||||
## Bare-Metal Deployment
|
||||
|
||||
Use this path when you want to manage PostgreSQL and Valkey yourself (e.g., existing infrastructure, managed cloud databases).
|
||||
|
||||
### Step 1 — Install system dependencies
|
||||
|
||||
```bash
|
||||
# Node.js 22 via nvm
|
||||
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
|
||||
nvm install 22
|
||||
nvm use 22
|
||||
|
||||
# pnpm
|
||||
npm install -g pnpm
|
||||
|
||||
# PostgreSQL 17 with pgvector (Debian/Ubuntu example)
|
||||
sudo apt-get install -y postgresql-17 postgresql-17-pgvector
|
||||
|
||||
# Valkey
|
||||
# Follow https://valkey.io/download/ for your distribution
|
||||
```
|
||||
|
||||
### Step 2 — Create the database
|
||||
|
||||
```sql
|
||||
-- Run as the postgres superuser
|
||||
CREATE USER mosaic WITH PASSWORD 'change-me';
|
||||
CREATE DATABASE mosaic OWNER mosaic;
|
||||
\c mosaic
|
||||
CREATE EXTENSION IF NOT EXISTS vector;
|
||||
```
|
||||
|
||||
### Step 3 — Clone and configure
|
||||
|
||||
```bash
|
||||
git clone <repo-url> /opt/mosaic
|
||||
cd /opt/mosaic
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Edit `/opt/mosaic/.env`. Required fields:
|
||||
|
||||
```dotenv
|
||||
DATABASE_URL=postgresql://mosaic:<password>@localhost:5432/mosaic
|
||||
VALKEY_URL=redis://localhost:6379
|
||||
BETTER_AUTH_SECRET=<openssl rand -base64 32>
|
||||
BETTER_AUTH_URL=https://your-domain.example.com
|
||||
GATEWAY_CORS_ORIGIN=https://your-domain.example.com
|
||||
NEXT_PUBLIC_GATEWAY_URL=https://your-domain.example.com
|
||||
```
|
||||
|
||||
### Step 4 — Install dependencies and build
|
||||
|
||||
```bash
|
||||
pnpm install
|
||||
pnpm build
|
||||
```
|
||||
|
||||
### Step 5 — Run database migrations
|
||||
|
||||
```bash
|
||||
pnpm --filter @mosaic/db db:migrate
|
||||
```
|
||||
|
||||
### Step 6 — Start the gateway
|
||||
|
||||
```bash
|
||||
node apps/gateway/dist/main.js
|
||||
```
|
||||
|
||||
The gateway reads `.env` from the monorepo root automatically (via `dotenv` in `main.ts`).
|
||||
|
||||
### Step 7 — Start the web app
|
||||
|
||||
```bash
|
||||
# Next.js standalone output
|
||||
node apps/web/.next/standalone/server.js
|
||||
```
|
||||
|
||||
The standalone build is self-contained; it does not require `node_modules` to be present at runtime.
|
||||
|
||||
### Step 8 — Configure a reverse proxy
|
||||
|
||||
#### Nginx example
|
||||
|
||||
```nginx
|
||||
# /etc/nginx/sites-available/mosaic
|
||||
|
||||
# Gateway API
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name your-domain.example.com;
|
||||
|
||||
ssl_certificate /etc/ssl/certs/your-domain.crt;
|
||||
ssl_certificate_key /etc/ssl/private/your-domain.key;
|
||||
|
||||
# WebSocket support (for chat.gateway.ts / Socket.IO)
|
||||
location /socket.io/ {
|
||||
proxy_pass http://127.0.0.1:4000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
}
|
||||
|
||||
# REST + auth
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:4000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
}
|
||||
|
||||
# Web app (optional — serve on a subdomain or a separate server block)
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name app.your-domain.example.com;
|
||||
|
||||
ssl_certificate /etc/ssl/certs/your-domain.crt;
|
||||
ssl_certificate_key /etc/ssl/private/your-domain.key;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:3000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Caddy example
|
||||
|
||||
```caddyfile
|
||||
# /etc/caddy/Caddyfile
|
||||
|
||||
your-domain.example.com {
|
||||
reverse_proxy /socket.io/* localhost:4000 {
|
||||
header_up Upgrade {http.upgrade}
|
||||
header_up Connection {http.connection}
|
||||
}
|
||||
reverse_proxy localhost:4000
|
||||
}
|
||||
|
||||
app.your-domain.example.com {
|
||||
reverse_proxy localhost:3000
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Production Considerations
|
||||
|
||||
### systemd Services
|
||||
|
||||
Create a service unit for each process.
|
||||
|
||||
**Gateway** — `/etc/systemd/system/mosaic-gateway.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Mosaic Gateway
|
||||
After=network.target postgresql.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=mosaic
|
||||
WorkingDirectory=/opt/mosaic
|
||||
EnvironmentFile=/opt/mosaic/.env
|
||||
ExecStart=/usr/bin/node apps/gateway/dist/main.js
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
**Web app** — `/etc/systemd/system/mosaic-web.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Mosaic Web App
|
||||
After=network.target mosaic-gateway.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=mosaic
|
||||
WorkingDirectory=/opt/mosaic/apps/web
|
||||
EnvironmentFile=/opt/mosaic/.env
|
||||
ExecStart=/usr/bin/node .next/standalone/server.js
|
||||
Environment=PORT=3000
|
||||
Environment=HOSTNAME=127.0.0.1
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
Enable and start:
|
||||
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now mosaic-gateway mosaic-web
|
||||
```
|
||||
|
||||
### Log Management
|
||||
|
||||
Gateway and web app logs go to systemd journal by default. View with:
|
||||
|
||||
```bash
|
||||
journalctl -u mosaic-gateway -f
|
||||
journalctl -u mosaic-web -f
|
||||
```
|
||||
|
||||
Rotate logs by configuring `journald` in `/etc/systemd/journald.conf`:
|
||||
|
||||
```ini
|
||||
SystemMaxUse=500M
|
||||
MaxRetentionSec=30day
|
||||
```
|
||||
|
||||
### Security Checklist
|
||||
|
||||
- Set `BETTER_AUTH_SECRET` to a cryptographically random value (`openssl rand -base64 32`).
|
||||
- Restrict `GATEWAY_CORS_ORIGIN` to your exact frontend origin — do not use `*`.
|
||||
- Run services as a dedicated non-root system user (e.g., `mosaic`).
|
||||
- Firewall: only expose ports 80/443 externally; keep 4000 and 3000 bound to `127.0.0.1`.
|
||||
- Set `AGENT_FILE_SANDBOX_DIR` to a directory outside the application root to prevent agent tools from accessing source code.
|
||||
- If using `AGENT_USER_TOOLS`, enumerate only the tools non-admin users need.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Gateway fails to start — "BETTER_AUTH_SECRET is required"
|
||||
|
||||
`BETTER_AUTH_SECRET` is missing or empty. Set it in `.env` and restart.
|
||||
|
||||
### `DATABASE_URL` connection refused
|
||||
|
||||
Verify PostgreSQL is running and the port matches. The Docker Compose default is `5433`; bare-metal typically uses `5432`.
|
||||
|
||||
```bash
|
||||
psql "$DATABASE_URL" -c '\conninfo'
|
||||
```
|
||||
|
||||
### pgvector extension missing
|
||||
|
||||
```sql
|
||||
\c mosaic
|
||||
CREATE EXTENSION IF NOT EXISTS vector;
|
||||
```
|
||||
|
||||
### Valkey / Redis connection refused
|
||||
|
||||
Check the URL in `VALKEY_URL`. The Docker Compose default is port `6380`.
|
||||
|
||||
```bash
|
||||
redis-cli -u "$VALKEY_URL" ping
|
||||
```
|
||||
|
||||
### WebSocket connections fail in production
|
||||
|
||||
Ensure your reverse proxy forwards the `Upgrade` and `Connection` headers. See the Nginx/Caddy examples above.
|
||||
|
||||
### Ollama models not appearing
|
||||
|
||||
Set `OLLAMA_BASE_URL` to the URL where Ollama is running (e.g., `http://localhost:11434`) and set `OLLAMA_MODELS` to a comma-separated list of model IDs you have pulled.
|
||||
|
||||
```bash
|
||||
ollama pull llama3.2
|
||||
```
|
||||
|
||||
### OTEL traces not appearing in Jaeger
|
||||
|
||||
Verify the collector is reachable at `OTEL_EXPORTER_OTLP_ENDPOINT`. With Docker Compose the default is `http://localhost:4318`. Check `docker compose ps` and `docker compose logs otel-collector`.
|
||||
|
||||
### Summarization / embedding features not working
|
||||
|
||||
These features require `OPENAI_API_KEY` to be set, or you must point `SUMMARIZATION_API_URL` / `EMBEDDING_API_URL` to an OpenAI-compatible endpoint (e.g., a local Ollama instance with an embeddings model).
|
||||
Reference in New Issue
Block a user