stack/docs/PORTAINER-DEPLOYMENT.md

# Portainer Deployment Guide

## Overview

This guide covers deploying Mosaic Stack via Portainer. Portainer has specific requirements that differ from standard docker-compose deployments.

## Key Differences

**Portainer Limitations:**

- ❌ Does NOT support `env_file:` directive
- ❌ Does NOT support localhost-only port bindings (`127.0.0.1:port`)
- ❌ Does NOT support health check conditions in `depends_on`
- ✅ DOES support environment variables defined in Portainer UI
- ✅ DOES support external networks

**Solution:** Use `docker-compose.portainer.yml` which is optimized for Portainer.

## Prerequisites

### 1. Create Overlay Network

**In Portainer:**

1. Go to **Networks** → **Add network**
2. Name: `mosaic_internal`
3. Driver: `overlay`
4. Enable **Attachable**
5. Click **Create network**

**Or via CLI:**

```bash
docker network create --driver=overlay --attachable mosaic_internal
```

### 2. Registry Authentication

If using private registry images from `git.mosaicstack.dev`:

**In Portainer:**

1. Go to **Registries** → **Add registry**
2. Type: **Custom Registry**
3. Name: `Gitea Mosaic`
4. Registry URL: `git.mosaicstack.dev`
5. Username: Your Gitea username
6. Password: Your Gitea password or access token
7. Click **Add registry**

## Deployment Steps

### Step 1: Deploy OpenBao Stack

**Why First?** OpenBao must be running before the main stack starts, as the API needs to connect to it.

**In Portainer:**

1. Go to **Stacks** → **Add stack**
2. **Name:** `mosaic-openbao`
3. **Build method:** Web editor
4. **Web editor:** Copy and paste contents of `docker-compose.portainer.yml`
5. **Environment variables:**
   ```
   IMAGE_TAG=latest
   OPENBAO_PORT=8200
   ```
6. Click **Deploy the stack**

**Verify Deployment:**

1. Go to **Stacks** → `mosaic-openbao`
2. Check both containers are running:
   - `mosaic-openbao` (should be running)
   - `mosaic-openbao-init` (will stop after initialization)
3. View logs:
   - Click on `mosaic-openbao-init` → **Logs**
   - Look for: `✓ OpenBao initialized successfully`

**Wait 30 seconds** for initialization to complete before proceeding.

### Step 2: Deploy Swarm Stack

**In Portainer:**

1. Go to **Stacks** → **Add stack**
2. **Name:** `mosaic`
3. **Build method:** Repository (recommended) or Web editor

**Option A: Git Repository (Recommended)**

- Repository URL: `https://git.mosaicstack.dev/mosaic/stack`
- Repository reference: `refs/heads/main`
- Compose path: `docker-compose.swarm.yml`
- Authentication: Enable if repository is private
- Enable **Automatic updates** (optional)
- Update interval: `5m` (checks every 5 minutes)

**Option B: Web Editor**

- Copy and paste contents of `docker-compose.swarm.yml`

4. **Environment variables:**

   ```
   IMAGE_TAG=latest
   POSTGRES_PASSWORD=<your-secure-password>
   JWT_SECRET=<your-jwt-secret>
   BETTER_AUTH_SECRET=<your-auth-secret>
   ENCRYPTION_KEY=<your-encryption-key>
   OIDC_CLIENT_ID=<your-oidc-client-id>
   OIDC_CLIENT_SECRET=<your-oidc-client-secret>
   OIDC_ISSUER=https://auth.diversecanvas.com/application/o/mosaic-stack/
   OIDC_REDIRECT_URI=https://api.mosaicstack.dev/auth/oauth2/callback/authentik
   OLLAMA_ENDPOINT=http://10.1.1.42:11434
   ```

5. Click **Deploy the stack**

### Step 3: Verify Deployment

**Check Services:**

1. Go to **Swarm** → **Services**
2. Verify all services show `1/1` replicas:
   - `mosaic_postgres`
   - `mosaic_valkey`
   - `mosaic_api`
   - `mosaic_web`
   - `mosaic_orchestrator`

**Check Logs:**

1. Go to **Swarm** → **Services** → `mosaic_api`
2. Click on the running task
3. Click **Logs**
4. Look for: `OpenBao connection established` or `Using fallback encryption`

**Access Services:**

- Web UI: http://app.mosaicstack.dev
- API: http://api.mosaicstack.dev/health
- OpenBao: http://<server-ip>:8200/ui

## Environment Variables Reference

### Required for All Deployments

```bash
# Image Configuration
IMAGE_TAG=latest                    # or 'latest' or specific commit SHA

# Database
POSTGRES_PASSWORD=<secure-password>
DATABASE_URL=postgresql://mosaic:${POSTGRES_PASSWORD}@postgres:5432/mosaic

# Security
JWT_SECRET=<random-32-chars>     # openssl rand -base64 32
BETTER_AUTH_SECRET=<random-32-chars>
ENCRYPTION_KEY=<64-char-hex>     # openssl rand -hex 32

# External OIDC (Authentik)
OIDC_CLIENT_ID=<from-authentik>
OIDC_CLIENT_SECRET=<from-authentik>
OIDC_ISSUER=https://auth.diversecanvas.com/application/o/mosaic-stack/
OIDC_REDIRECT_URI=https://api.mosaicstack.dev/auth/oauth2/callback/authentik

# External Ollama
OLLAMA_ENDPOINT=http://10.1.1.42:11434
```

### Optional Variables

```bash
# Ports
OPENBAO_PORT=8200
API_PORT=3001
WEB_PORT=3000

# Cache
VALKEY_MAXMEMORY=256mb

# Logging
LOG_LEVEL=info
DEBUG=false
```

## Updating Stacks

### Update OpenBao

1. Go to **Stacks** → `mosaic-openbao`
2. Click **Editor**
3. Update `IMAGE_TAG` environment variable (or compose file)
4. Click **Update the stack**
5. Enable **Re-pull image**
6. Click **Update**

### Update Swarm Stack

**If using Git Repository:**

- Portainer automatically pulls updates based on update interval
- Or manually: **Stacks** → `mosaic` → **Pull and redeploy**

**If using Web Editor:**

1. Go to **Stacks** → `mosaic`
2. Click **Editor**
3. Update compose file or environment variables
4. Click **Update the stack**
5. Enable **Re-pull images**
6. Click **Update**

## Troubleshooting

### Stack Deployment Failed: "network not found"

**Cause:** `mosaic_internal` network doesn't exist

**Fix:**

```bash
docker network create --driver=overlay --attachable mosaic_internal
```

### OpenBao Unhealthy

**Check logs:**

1. **Containers** → `mosaic-openbao` → **Logs**
2. Look for errors

**Common issues:**

- Port 8200 already in use → Stop conflicting service
- Config file not found → Rebuild image
- Permission denied → Check volume permissions

### API Can't Connect to OpenBao

**Symptoms:**

- API logs show "connection refused" to OpenBao
- API falls back to encryption key

**Check:**

1. **Containers** → `mosaic-openbao` → Verify it's running
2. **Networks** → `mosaic_internal` → Verify both stacks are connected
3. API environment: `OPENBAO_ADDR=http://openbao:8200`

**Test connectivity:**

1. **Containers** → Find API container → **Console**
2. Run: `wget -qO- http://openbao:8200/v1/sys/health`

### Services Not Starting in Swarm Stack

**Check service logs:**

1. **Swarm** → **Services** → Click service
2. Click running/failed task
3. View logs

**Common issues:**

- Missing environment variables
- Image pull failed (check registry authentication)
- Dependency not ready (database, OpenBao)

## Best Practices

### Security

1. **Never expose OpenBao to public internet** without TLS
2. **Use firewall rules** to restrict port 8200 access
3. **Rotate secrets regularly** (JWT_SECRET, API keys)
4. **Use strong passwords** for PostgreSQL
5. **Enable TLS** for production deployments

### Monitoring

**In Portainer:**

1. **Dashboard** → View resource usage
2. **Stacks** → Monitor stack health
3. **Swarm** → **Services** → Monitor replicas

**Set up alerts:**

1. **Notifications** → Configure webhook/email
2. **Events** → Enable service alerts

### Backups

**OpenBao Volumes:**

```bash
# Backup
docker run --rm \
  -v mosaic-openbao-data:/data \
  -v $(pwd):/backup \
  alpine tar czf /backup/openbao-$(date +%Y%m%d).tar.gz -C /data .

# Restore
docker run --rm \
  -v mosaic-openbao-data:/data \
  -v $(pwd):/backup \
  alpine sh -c 'rm -rf /data/* && tar xzf /backup/openbao-20260208.tar.gz -C /data'
```

**PostgreSQL:**

```bash
# Backup
docker exec mosaic_postgres pg_dump -U mosaic mosaic > backup.sql

# Restore
docker exec -i mosaic_postgres psql -U mosaic mosaic < backup.sql
```

## Advanced Configuration

### Using External Services

To use external PostgreSQL, Valkey, or Vault:

1. Edit `docker-compose.swarm.yml` in Portainer
2. Comment out the service you want to replace
3. Update environment variables to point to external service
4. Update the stack

**Example - External PostgreSQL:**

```yaml
# Comment out postgres service
# postgres:
#   ...

# Update API environment
services:
  api:
    environment:
      DATABASE_URL: postgresql://user:pass@external-db.example.com:5432/mosaic
```

### Custom Domains

Update environment variables:

```bash
NEXT_PUBLIC_APP_URL=https://mosaic.example.com
NEXT_PUBLIC_API_URL=https://api.example.com
OIDC_REDIRECT_URI=https://api.example.com/auth/oauth2/callback/authentik
```

### Resource Limits

Add to services in Portainer editor:

```yaml
services:
  api:
    deploy:
      resources:
        limits:
          cpus: "1"
          memory: 1G
        reservations:
          cpus: "0.5"
          memory: 512M
```

## See Also

- [Swarm Deployment Guide](SWARM-DEPLOYMENT.md) - CLI-based deployment
- [OpenBao Deployment Guide](OPENBAO-DEPLOYMENT.md) - OpenBao options
- [Configuration Guide](CONFIGURATION.md) - All environment variables
- [Docker Compose Guide](../docker/DOCKER-COMPOSE-GUIDE.md) - Standalone deployment