feat(#37-41): Add domains, ideas, relationships, agents, widgets schema
Schema additions for issues #37-41: New models: - Domain (#37): Life domains (work, marriage, homelab, etc.) - Idea (#38): Brain dumps with pgvector embeddings - Relationship (#39): Generic entity linking (blocks, depends_on) - Agent (#40): ClawdBot agent tracking with metrics - AgentSession (#40): Conversation session tracking - WidgetDefinition (#41): HUD widget registry - UserLayout (#41): Per-user dashboard configuration Updated models: - Task, Event, Project: Added domainId foreign key - User, Workspace: Added new relations New enums: - IdeaStatus: CAPTURED, PROCESSING, ACTIONABLE, ARCHIVED, DISCARDED - RelationshipType: BLOCKS, BLOCKED_BY, DEPENDS_ON, etc. - AgentStatus: IDLE, WORKING, WAITING, ERROR, TERMINATED - EntityType: Added IDEA, DOMAIN Migration: 20260129182803_add_domains_ideas_agents_widgets
This commit is contained in:
Jason Woltje
349
docs/1-getting-started/4-docker-deployment/QUICK-REFERENCE.md
Normal file
349
docs/1-getting-started/4-docker-deployment/QUICK-REFERENCE.md
Normal file
@@ -0,0 +1,349 @@
|
||||
# Docker Quick Reference
|
||||
|
||||
Quick command reference for Mosaic Stack Docker operations.
|
||||
|
||||
## Starting Services
|
||||
|
||||
```bash
|
||||
# Core services only (PostgreSQL, Valkey, API, Web)
|
||||
docker compose up -d
|
||||
|
||||
# With Authentik OIDC
|
||||
docker compose --profile authentik up -d
|
||||
|
||||
# With Ollama AI
|
||||
docker compose --profile ollama up -d
|
||||
|
||||
# All services
|
||||
docker compose --profile full up -d
|
||||
|
||||
# Or use Makefile
|
||||
make docker-up # Core services
|
||||
make docker-up-full # All services
|
||||
```
|
||||
|
||||
## Stopping Services
|
||||
|
||||
```bash
|
||||
# Stop (keeps data)
|
||||
docker compose down
|
||||
|
||||
# Stop and remove volumes (deletes data!)
|
||||
docker compose down -v
|
||||
```
|
||||
|
||||
## Viewing Status
|
||||
|
||||
```bash
|
||||
# List running services
|
||||
docker compose ps
|
||||
|
||||
# View logs (all services)
|
||||
docker compose logs -f
|
||||
|
||||
# View logs (specific service)
|
||||
docker compose logs -f api
|
||||
|
||||
# Last 100 lines
|
||||
docker compose logs --tail=100 api
|
||||
|
||||
# Resource usage
|
||||
docker stats
|
||||
```
|
||||
|
||||
## Service Management
|
||||
|
||||
```bash
|
||||
# Restart all services
|
||||
docker compose restart
|
||||
|
||||
# Restart specific service
|
||||
docker compose restart api
|
||||
|
||||
# Rebuild and restart
|
||||
docker compose up -d --build
|
||||
|
||||
# Pull latest images
|
||||
docker compose pull
|
||||
```
|
||||
|
||||
## Database Operations
|
||||
|
||||
```bash
|
||||
# PostgreSQL shell
|
||||
docker compose exec postgres psql -U mosaic -d mosaic
|
||||
|
||||
# Run migrations
|
||||
docker compose exec api pnpm prisma:migrate:prod
|
||||
|
||||
# Seed data
|
||||
docker compose exec api pnpm prisma:seed
|
||||
|
||||
# Backup database
|
||||
docker compose exec postgres pg_dump -U mosaic mosaic > backup.sql
|
||||
|
||||
# Restore database
|
||||
cat backup.sql | docker compose exec -T postgres psql -U mosaic -d mosaic
|
||||
```
|
||||
|
||||
## Cache Operations
|
||||
|
||||
```bash
|
||||
# Valkey CLI
|
||||
docker compose exec valkey valkey-cli
|
||||
|
||||
# Check cache health
|
||||
docker compose exec valkey valkey-cli ping
|
||||
|
||||
# Flush cache
|
||||
docker compose exec valkey valkey-cli FLUSHALL
|
||||
```
|
||||
|
||||
## Container Access
|
||||
|
||||
```bash
|
||||
# API container shell
|
||||
docker compose exec api sh
|
||||
|
||||
# Web container shell
|
||||
docker compose exec web sh
|
||||
|
||||
# Run command in container
|
||||
docker compose exec api node -v
|
||||
```
|
||||
|
||||
## Debugging
|
||||
|
||||
```bash
|
||||
# Check service health
|
||||
docker compose ps
|
||||
|
||||
# View container details
|
||||
docker inspect mosaic-api
|
||||
|
||||
# Check networks
|
||||
docker network ls
|
||||
docker network inspect mosaic-internal
|
||||
|
||||
# Check volumes
|
||||
docker volume ls
|
||||
docker volume inspect mosaic-postgres-data
|
||||
|
||||
# Test connectivity
|
||||
docker compose exec api nc -zv postgres 5432
|
||||
docker compose exec api nc -zv valkey 6379
|
||||
```
|
||||
|
||||
## Health Endpoints
|
||||
|
||||
```bash
|
||||
# API health
|
||||
curl http://localhost:3001/health
|
||||
|
||||
# Web (returns HTML)
|
||||
curl http://localhost:3000
|
||||
|
||||
# Authentik health (if enabled)
|
||||
curl http://localhost:9000/-/health/live/
|
||||
|
||||
# Ollama health (if enabled)
|
||||
curl http://localhost:11434/api/tags
|
||||
```
|
||||
|
||||
## Cleanup
|
||||
|
||||
```bash
|
||||
# Remove stopped containers
|
||||
docker compose down
|
||||
|
||||
# Remove containers and volumes
|
||||
docker compose down -v
|
||||
|
||||
# Remove all unused Docker resources
|
||||
docker system prune -a
|
||||
|
||||
# Remove unused volumes only
|
||||
docker volume prune
|
||||
```
|
||||
|
||||
## Environment Management
|
||||
|
||||
```bash
|
||||
# Check current environment
|
||||
docker compose config
|
||||
|
||||
# Validate compose file
|
||||
docker compose config --quiet
|
||||
|
||||
# Use specific env file
|
||||
docker compose --env-file .env.production up -d
|
||||
```
|
||||
|
||||
## Profiles
|
||||
|
||||
```bash
|
||||
# Available profiles
|
||||
# - authentik: Authentik OIDC stack
|
||||
# - ollama: Ollama AI service
|
||||
# - full: All optional services
|
||||
|
||||
# Set profile via environment
|
||||
export COMPOSE_PROFILES=authentik,ollama
|
||||
docker compose up -d
|
||||
|
||||
# Or in .env file
|
||||
echo "COMPOSE_PROFILES=full" >> .env
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
```bash
|
||||
# Container won't start - check logs
|
||||
docker compose logs <service>
|
||||
|
||||
# Port conflict - check what's using the port
|
||||
lsof -i :3001
|
||||
|
||||
# Permission errors - check permissions
|
||||
docker compose exec postgres ls -la /var/lib/postgresql/data
|
||||
|
||||
# Network issues - recreate networks
|
||||
docker compose down
|
||||
docker network prune
|
||||
docker compose up -d
|
||||
|
||||
# Volume issues - check volume
|
||||
docker volume inspect mosaic-postgres-data
|
||||
|
||||
# Reset everything (DANGER: deletes all data)
|
||||
docker compose down -v
|
||||
docker system prune -af
|
||||
docker volume prune -f
|
||||
```
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
```bash
|
||||
# View resource usage
|
||||
docker stats
|
||||
|
||||
# Limit container resources (in docker-compose.override.yml)
|
||||
services:
|
||||
api:
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '1.0'
|
||||
memory: 1G
|
||||
|
||||
# Adjust PostgreSQL settings in .env
|
||||
POSTGRES_SHARED_BUFFERS=512MB
|
||||
POSTGRES_EFFECTIVE_CACHE_SIZE=2GB
|
||||
|
||||
# Adjust Valkey memory in .env
|
||||
VALKEY_MAXMEMORY=512mb
|
||||
```
|
||||
|
||||
## Backup & Restore
|
||||
|
||||
```bash
|
||||
# Backup PostgreSQL database
|
||||
docker compose exec postgres pg_dump -U mosaic mosaic > backup-$(date +%Y%m%d).sql
|
||||
|
||||
# Backup volume
|
||||
docker run --rm \
|
||||
-v mosaic-postgres-data:/data \
|
||||
-v $(pwd):/backup \
|
||||
alpine tar czf /backup/postgres-backup.tar.gz /data
|
||||
|
||||
# Restore database
|
||||
cat backup-20260128.sql | docker compose exec -T postgres psql -U mosaic -d mosaic
|
||||
|
||||
# Restore volume
|
||||
docker run --rm \
|
||||
-v mosaic-postgres-data:/data \
|
||||
-v $(pwd):/backup \
|
||||
alpine tar xzf /backup/postgres-backup.tar.gz -C /
|
||||
```
|
||||
|
||||
## Security
|
||||
|
||||
```bash
|
||||
# Scan images for vulnerabilities
|
||||
docker scout cves mosaic-api
|
||||
|
||||
# Check running processes in container
|
||||
docker compose exec api ps aux
|
||||
|
||||
# View container security options
|
||||
docker inspect mosaic-api --format='{{.HostConfig.SecurityOpt}}'
|
||||
|
||||
# Rotate secrets (update .env, then)
|
||||
docker compose up -d --force-recreate
|
||||
```
|
||||
|
||||
## Makefile Commands
|
||||
|
||||
```bash
|
||||
make help # Show all commands
|
||||
make docker-up # Start core services
|
||||
make docker-up-full # Start all services
|
||||
make docker-down # Stop services
|
||||
make docker-logs # View logs
|
||||
make docker-ps # Service status
|
||||
make docker-build # Rebuild images
|
||||
make docker-restart # Restart services
|
||||
make docker-test # Run smoke test
|
||||
make docker-clean # Remove containers and volumes
|
||||
```
|
||||
|
||||
## npm Scripts
|
||||
|
||||
```bash
|
||||
pnpm docker:up # Start services
|
||||
pnpm docker:down # Stop services
|
||||
pnpm docker:logs # View logs
|
||||
pnpm docker:ps # Service status
|
||||
pnpm docker:build # Rebuild images
|
||||
pnpm docker:restart # Restart services
|
||||
pnpm test:docker # Run integration tests
|
||||
```
|
||||
|
||||
## One-Liners
|
||||
|
||||
```bash
|
||||
# Quick health check all services
|
||||
docker compose ps | grep -E 'Up|healthy'
|
||||
|
||||
# Follow logs for all services with timestamps
|
||||
docker compose logs -f --timestamps
|
||||
|
||||
# Restart unhealthy services
|
||||
docker compose ps --format json | jq -r 'select(.Health == "unhealthy") | .Service' | xargs docker compose restart
|
||||
|
||||
# Show disk usage by service
|
||||
docker system df -v
|
||||
|
||||
# Export all logs to file
|
||||
docker compose logs > logs-$(date +%Y%m%d-%H%M%S).txt
|
||||
|
||||
# Check which ports are exposed
|
||||
docker compose ps --format json | jq -r '.[] | "\(.Service): \(.Publishers)"'
|
||||
```
|
||||
|
||||
## Service URLs (Default Ports)
|
||||
|
||||
- **Web**: http://localhost:3000
|
||||
- **API**: http://localhost:3001
|
||||
- **API Health**: http://localhost:3001/health
|
||||
- **PostgreSQL**: localhost:5432
|
||||
- **Valkey**: localhost:6379
|
||||
- **Authentik**: http://localhost:9000 (if enabled)
|
||||
- **Ollama**: http://localhost:11434 (if enabled)
|
||||
- **Prisma Studio**: http://localhost:5555 (when running)
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Full Deployment Guide](README.md)
|
||||
- [Configuration Reference](../3-configuration/3-docker.md)
|
||||
- [Troubleshooting Guide](README.md#troubleshooting)
|
||||
416
docs/1-getting-started/4-docker-deployment/README.md
Normal file
416
docs/1-getting-started/4-docker-deployment/README.md
Normal file
@@ -0,0 +1,416 @@
|
||||
# Docker Deployment
|
||||
|
||||
Complete guide for deploying Mosaic Stack using Docker Compose.
|
||||
|
||||
## Overview
|
||||
|
||||
Mosaic Stack provides a turnkey Docker Compose setup that includes:
|
||||
|
||||
- **PostgreSQL 17** with pgvector extension
|
||||
- **Valkey** (Redis-compatible cache)
|
||||
- **Traefik** (optional reverse proxy)
|
||||
- **Authentik** (optional OIDC provider)
|
||||
- **Ollama** (optional AI service)
|
||||
- **Mosaic API** (NestJS backend)
|
||||
- **Mosaic Web** (Next.js frontend)
|
||||
|
||||
## Quick Start
|
||||
|
||||
Start the entire stack with one command:
|
||||
|
||||
```bash
|
||||
# Copy environment configuration
|
||||
cp .env.example .env
|
||||
|
||||
# Edit .env with your settings
|
||||
nano .env
|
||||
|
||||
# Start core services (PostgreSQL, Valkey, API, Web)
|
||||
docker compose up -d
|
||||
|
||||
# Check service status
|
||||
docker compose ps
|
||||
|
||||
# View logs
|
||||
docker compose logs -f
|
||||
```
|
||||
|
||||
That's it! Your Mosaic Stack is now running:
|
||||
- Web: http://localhost:3000
|
||||
- API: http://localhost:3001
|
||||
- PostgreSQL: localhost:5432
|
||||
- Valkey: localhost:6379
|
||||
|
||||
## Service Profiles
|
||||
|
||||
Mosaic Stack uses Docker Compose profiles to enable optional services:
|
||||
|
||||
### Core Services (Always Active)
|
||||
- `postgres` - PostgreSQL database
|
||||
- `valkey` - Valkey cache
|
||||
- `api` - Mosaic API
|
||||
- `web` - Mosaic Web
|
||||
|
||||
### Optional Services (Profiles)
|
||||
|
||||
#### Traefik (Reverse Proxy)
|
||||
```bash
|
||||
# Start with bundled Traefik
|
||||
docker compose --profile traefik-bundled up -d
|
||||
|
||||
# Or set in .env
|
||||
COMPOSE_PROFILES=traefik-bundled
|
||||
```
|
||||
|
||||
Services included:
|
||||
- `traefik` - Traefik reverse proxy with dashboard (http://localhost:8080)
|
||||
|
||||
See [Traefik Integration Guide](traefik.md) for detailed configuration options including upstream mode.
|
||||
|
||||
#### Authentik (OIDC Provider)
|
||||
```bash
|
||||
# Start with Authentik
|
||||
docker compose --profile authentik up -d
|
||||
|
||||
# Or set in .env
|
||||
COMPOSE_PROFILES=authentik
|
||||
```
|
||||
|
||||
Services included:
|
||||
- `authentik-postgres` - Authentik database
|
||||
- `authentik-redis` - Authentik cache
|
||||
- `authentik-server` - Authentik OIDC server (http://localhost:9000)
|
||||
- `authentik-worker` - Authentik background worker
|
||||
|
||||
#### Ollama (AI Service)
|
||||
```bash
|
||||
# Start with Ollama
|
||||
docker compose --profile ollama up -d
|
||||
|
||||
# Or set in .env
|
||||
COMPOSE_PROFILES=ollama
|
||||
```
|
||||
|
||||
Services included:
|
||||
- `ollama` - Ollama LLM service (http://localhost:11434)
|
||||
|
||||
#### All Services
|
||||
```bash
|
||||
# Start everything
|
||||
docker compose --profile full up -d
|
||||
|
||||
# Or set in .env
|
||||
COMPOSE_PROFILES=full
|
||||
```
|
||||
|
||||
## Deployment Modes
|
||||
|
||||
### Turnkey Deployment (Recommended for Development)
|
||||
|
||||
Uses all bundled services:
|
||||
|
||||
```bash
|
||||
# Start core services
|
||||
docker compose up -d
|
||||
|
||||
# Or with optional services
|
||||
docker compose --profile full up -d
|
||||
```
|
||||
|
||||
### Customized Deployment (Production)
|
||||
|
||||
Use external services for production:
|
||||
|
||||
1. Copy override template:
|
||||
```bash
|
||||
cp docker-compose.override.yml.example docker-compose.override.yml
|
||||
```
|
||||
|
||||
2. Edit `docker-compose.override.yml` to:
|
||||
- Disable bundled services
|
||||
- Point to external services
|
||||
- Add custom configuration
|
||||
|
||||
3. Start stack:
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Docker automatically merges `docker-compose.yml` and `docker-compose.override.yml`.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Network Configuration
|
||||
|
||||
Mosaic Stack uses two Docker networks to organize service communication:
|
||||
|
||||
#### mosaic-internal (Backend Services)
|
||||
- **Purpose**: Isolates database and cache services
|
||||
- **Services**:
|
||||
- PostgreSQL (main database)
|
||||
- Valkey (main cache)
|
||||
- Authentik PostgreSQL
|
||||
- Authentik Redis
|
||||
- Ollama (when using bundled service)
|
||||
- **Connectivity**:
|
||||
- Not marked as `internal: true` to allow API to reach external services
|
||||
- API can connect to external Authentik and Ollama instances
|
||||
- Database and cache services only accessible within Docker network
|
||||
- No direct external access to database/cache ports (unless explicitly exposed)
|
||||
|
||||
#### mosaic-public (Frontend Services)
|
||||
- **Purpose**: Services that need external network access
|
||||
- **Services**:
|
||||
- Mosaic API (needs to reach Authentik OIDC and external Ollama)
|
||||
- Mosaic Web
|
||||
- Authentik Server (receives OIDC callbacks)
|
||||
- **Connectivity**: Full external network access for API integrations
|
||||
|
||||
#### Network Security Notes
|
||||
|
||||
1. **Why mosaic-internal is not marked internal**: The API service needs to:
|
||||
- Connect to external Authentik servers for OIDC authentication
|
||||
- Connect to external Ollama services when using remote AI
|
||||
- Make outbound HTTP requests for integrations
|
||||
|
||||
2. **Database/Cache Protection**: Even though the network allows external access:
|
||||
- PostgreSQL and Valkey are NOT exposed on host ports by default
|
||||
- Only accessible via internal Docker DNS (postgres:5432, valkey:6379)
|
||||
- To expose for development, explicitly set ports in `.env`
|
||||
|
||||
3. **Production Recommendations**:
|
||||
- Use firewall rules to restrict container egress traffic
|
||||
- Use reverse proxy (Traefik, nginx) for API/Web with TLS
|
||||
- Use external managed PostgreSQL and Valkey services
|
||||
- Implement network policies in orchestration platforms (Kubernetes)
|
||||
|
||||
### Volume Management
|
||||
|
||||
Persistent data volumes:
|
||||
|
||||
- `mosaic-postgres-data` - PostgreSQL data
|
||||
- `mosaic-valkey-data` - Valkey persistence
|
||||
- `mosaic-authentik-postgres-data` - Authentik database
|
||||
- `mosaic-authentik-redis-data` - Authentik cache
|
||||
- `mosaic-authentik-media` - Authentik media files
|
||||
- `mosaic-authentik-certs` - Authentik certificates
|
||||
- `mosaic-authentik-templates` - Authentik templates
|
||||
- `mosaic-ollama-data` - Ollama models
|
||||
|
||||
### Health Checks
|
||||
|
||||
All services include health checks with automatic restart:
|
||||
|
||||
- PostgreSQL: `pg_isready` check every 10s
|
||||
- Valkey: `valkey-cli ping` every 10s
|
||||
- API: HTTP GET /health every 30s
|
||||
- Web: HTTP GET / every 30s
|
||||
- Authentik: HTTP GET /-/health/live/ every 30s
|
||||
|
||||
## Common Operations
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
# All services
|
||||
docker compose logs -f
|
||||
|
||||
# Specific service
|
||||
docker compose logs -f api
|
||||
|
||||
# Last 100 lines
|
||||
docker compose logs --tail=100 api
|
||||
```
|
||||
|
||||
### Restart Services
|
||||
|
||||
```bash
|
||||
# Restart all
|
||||
docker compose restart
|
||||
|
||||
# Restart specific service
|
||||
docker compose restart api
|
||||
```
|
||||
|
||||
### Stop Services
|
||||
|
||||
```bash
|
||||
# Stop all (keeps data)
|
||||
docker compose down
|
||||
|
||||
# Stop and remove volumes (WARNING: deletes all data)
|
||||
docker compose down -v
|
||||
```
|
||||
|
||||
### Execute Commands in Containers
|
||||
|
||||
```bash
|
||||
# PostgreSQL shell
|
||||
docker compose exec postgres psql -U mosaic -d mosaic
|
||||
|
||||
# API shell
|
||||
docker compose exec api sh
|
||||
|
||||
# Run Prisma migrations
|
||||
docker compose exec api pnpm prisma:migrate:prod
|
||||
```
|
||||
|
||||
### Update Services
|
||||
|
||||
```bash
|
||||
# Pull latest images
|
||||
docker compose pull
|
||||
|
||||
# Rebuild and restart
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
See [Configuration Guide](../3-configuration/README.md) for detailed environment variable documentation.
|
||||
|
||||
### Key Environment Variables
|
||||
|
||||
```bash
|
||||
# Application Ports
|
||||
API_PORT=3001
|
||||
WEB_PORT=3000
|
||||
|
||||
# Database
|
||||
DATABASE_URL=postgresql://mosaic:password@postgres:5432/mosaic
|
||||
POSTGRES_USER=mosaic
|
||||
POSTGRES_PASSWORD=change-me
|
||||
|
||||
# Cache
|
||||
VALKEY_URL=redis://valkey:6379
|
||||
|
||||
# Authentication (if using Authentik)
|
||||
OIDC_ISSUER=https://auth.example.com/application/o/mosaic-stack/
|
||||
OIDC_CLIENT_ID=your-client-id
|
||||
OIDC_CLIENT_SECRET=your-client-secret
|
||||
|
||||
# JWT
|
||||
JWT_SECRET=change-this-to-a-random-secret
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Service Won't Start
|
||||
|
||||
Check logs:
|
||||
```bash
|
||||
docker compose logs <service-name>
|
||||
```
|
||||
|
||||
Common issues:
|
||||
- Port already in use: Change port in `.env`
|
||||
- Health check failing: Wait longer or check service logs
|
||||
- Missing environment variables: Check `.env` file
|
||||
|
||||
### Database Connection Issues
|
||||
|
||||
1. Verify PostgreSQL is healthy:
|
||||
```bash
|
||||
docker compose ps postgres
|
||||
```
|
||||
|
||||
2. Check database logs:
|
||||
```bash
|
||||
docker compose logs postgres
|
||||
```
|
||||
|
||||
3. Test connection:
|
||||
```bash
|
||||
docker compose exec postgres psql -U mosaic -d mosaic -c "SELECT 1;"
|
||||
```
|
||||
|
||||
### Performance Issues
|
||||
|
||||
1. Adjust PostgreSQL settings in `.env`:
|
||||
```bash
|
||||
POSTGRES_SHARED_BUFFERS=512MB
|
||||
POSTGRES_EFFECTIVE_CACHE_SIZE=2GB
|
||||
POSTGRES_MAX_CONNECTIONS=200
|
||||
```
|
||||
|
||||
2. Adjust Valkey memory:
|
||||
```bash
|
||||
VALKEY_MAXMEMORY=512mb
|
||||
```
|
||||
|
||||
3. Check resource usage:
|
||||
```bash
|
||||
docker stats
|
||||
```
|
||||
|
||||
### Reset Everything
|
||||
|
||||
```bash
|
||||
# Stop and remove all containers, networks, and volumes
|
||||
docker compose down -v
|
||||
|
||||
# Remove all Mosaic images
|
||||
docker images | grep mosaic | awk '{print $3}' | xargs docker rmi
|
||||
|
||||
# Start fresh
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
## Production Considerations
|
||||
|
||||
### Security
|
||||
|
||||
1. **Change default passwords** in `.env`:
|
||||
- `POSTGRES_PASSWORD`
|
||||
- `AUTHENTIK_POSTGRES_PASSWORD`
|
||||
- `AUTHENTIK_SECRET_KEY`
|
||||
- `JWT_SECRET`
|
||||
|
||||
2. **Use secrets management**:
|
||||
- Docker secrets
|
||||
- External secret manager (Vault, AWS Secrets Manager)
|
||||
|
||||
3. **Network security**:
|
||||
- Use reverse proxy (see [Traefik Integration](traefik.md))
|
||||
- Enable HTTPS/TLS
|
||||
- Restrict port exposure
|
||||
|
||||
4. **Regular updates**:
|
||||
- Keep images updated
|
||||
- Monitor security advisories
|
||||
|
||||
### Backup
|
||||
|
||||
Backup volumes regularly:
|
||||
|
||||
```bash
|
||||
# Backup PostgreSQL
|
||||
docker compose exec postgres pg_dump -U mosaic mosaic > backup.sql
|
||||
|
||||
# Backup volumes
|
||||
docker run --rm -v mosaic-postgres-data:/data -v $(pwd):/backup alpine tar czf /backup/postgres-data.tar.gz /data
|
||||
```
|
||||
|
||||
### Monitoring
|
||||
|
||||
Consider adding:
|
||||
- Prometheus for metrics
|
||||
- Grafana for dashboards
|
||||
- Loki for log aggregation
|
||||
- Alertmanager for alerts
|
||||
|
||||
### Scaling
|
||||
|
||||
For production scaling:
|
||||
- Use external PostgreSQL (managed service)
|
||||
- Use external Redis/Valkey cluster
|
||||
- Load balance multiple API instances
|
||||
- Use CDN for static assets
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Traefik Integration](traefik.md) - Reverse proxy setup and configuration
|
||||
- [Installation Guide](../2-installation/README.md) - Detailed setup instructions
|
||||
- [Configuration Guide](../3-configuration/README.md) - Environment variables
|
||||
- [Development Guide](../../2-development/README.md) - Development workflow
|
||||
- [API Documentation](../../4-api/README.md) - API reference
|
||||
521
docs/1-getting-started/4-docker-deployment/traefik.md
Normal file
521
docs/1-getting-started/4-docker-deployment/traefik.md
Normal file
@@ -0,0 +1,521 @@
|
||||
# Traefik Reverse Proxy Integration
|
||||
|
||||
Mosaic Stack supports flexible Traefik integration with three deployment modes:
|
||||
|
||||
1. **Bundled Mode**: Self-contained Traefik instance
|
||||
2. **Upstream Mode**: Connect to external Traefik
|
||||
3. **None Mode**: Direct port exposure (no reverse proxy)
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Bundled Mode (Recommended for New Deployments)
|
||||
|
||||
```bash
|
||||
# 1. Copy bundled configuration
|
||||
cp .env.traefik-bundled.example .env
|
||||
|
||||
# 2. Update passwords and secrets in .env
|
||||
# Edit POSTGRES_PASSWORD, AUTHENTIK_SECRET_KEY, JWT_SECRET, etc.
|
||||
|
||||
# 3. Start with bundled Traefik profile
|
||||
docker compose --profile traefik-bundled up -d
|
||||
|
||||
# 4. Access services
|
||||
# - Web: https://mosaic.local
|
||||
# - API: https://api.mosaic.local
|
||||
# - Auth: https://auth.mosaic.local
|
||||
# - Traefik Dashboard: http://localhost:8080
|
||||
```
|
||||
|
||||
### Upstream Mode (For Existing Traefik Instances)
|
||||
|
||||
```bash
|
||||
# 1. Ensure external Traefik network exists
|
||||
docker network create traefik-public
|
||||
|
||||
# 2. Copy upstream configuration
|
||||
cp .env.traefik-upstream.example .env
|
||||
|
||||
# 3. Update .env with your domains and secrets
|
||||
|
||||
# 4. Create override file for network attachment
|
||||
cp docker-compose.override.yml.example docker-compose.override.yml
|
||||
|
||||
# 5. Edit docker-compose.override.yml and uncomment upstream network section
|
||||
|
||||
# 6. Start services
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### None Mode (Direct Port Access)
|
||||
|
||||
```bash
|
||||
# 1. Use default .env.example
|
||||
cp .env.example .env
|
||||
|
||||
# 2. Ensure TRAEFIK_MODE=none (default)
|
||||
|
||||
# 3. Start services
|
||||
docker compose up -d
|
||||
|
||||
# 4. Access services on standard ports
|
||||
# - Web: http://localhost:3000
|
||||
# - API: http://localhost:3001
|
||||
# - Auth: http://localhost:9000
|
||||
```
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `TRAEFIK_MODE` | `none` | Traefik mode: `bundled`, `upstream`, or `none` |
|
||||
| `TRAEFIK_ENABLE` | `false` | Enable Traefik labels on services |
|
||||
| `MOSAIC_API_DOMAIN` | `api.mosaic.local` | Domain for API service |
|
||||
| `MOSAIC_WEB_DOMAIN` | `mosaic.local` | Domain for Web service |
|
||||
| `MOSAIC_AUTH_DOMAIN` | `auth.mosaic.local` | Domain for Authentik service |
|
||||
| `TRAEFIK_NETWORK` | `traefik-public` | External Traefik network (upstream mode) |
|
||||
| `TRAEFIK_TLS_ENABLED` | `true` | Enable TLS/HTTPS |
|
||||
| `TRAEFIK_ACME_EMAIL` | - | Email for Let's Encrypt (production) |
|
||||
| `TRAEFIK_CERTRESOLVER` | - | Cert resolver name (e.g., `letsencrypt`) |
|
||||
| `TRAEFIK_DASHBOARD_ENABLED` | `true` | Enable Traefik dashboard (bundled mode) |
|
||||
| `TRAEFIK_DASHBOARD_PORT` | `8080` | Dashboard port (bundled mode) |
|
||||
| `TRAEFIK_ENTRYPOINT` | `websecure` | Traefik entrypoint (`web` or `websecure`) |
|
||||
| `TRAEFIK_DOCKER_NETWORK` | `mosaic-public` | Docker network for Traefik routing |
|
||||
|
||||
### Docker Compose Profiles
|
||||
|
||||
| Profile | Description |
|
||||
|---------|-------------|
|
||||
| `traefik-bundled` | Activates bundled Traefik service |
|
||||
| `authentik` | Enables Authentik SSO services |
|
||||
| `ollama` | Enables Ollama AI service |
|
||||
| `full` | Enables all optional services |
|
||||
|
||||
## Deployment Scenarios
|
||||
|
||||
### Local Development with Self-Signed Certificates
|
||||
|
||||
```bash
|
||||
# .env configuration
|
||||
TRAEFIK_MODE=bundled
|
||||
TRAEFIK_ENABLE=true
|
||||
TRAEFIK_TLS_ENABLED=true
|
||||
TRAEFIK_ACME_EMAIL= # Empty for self-signed
|
||||
MOSAIC_API_DOMAIN=api.mosaic.local
|
||||
MOSAIC_WEB_DOMAIN=mosaic.local
|
||||
MOSAIC_AUTH_DOMAIN=auth.mosaic.local
|
||||
|
||||
# Start services
|
||||
docker compose --profile traefik-bundled up -d
|
||||
|
||||
# Add to /etc/hosts
|
||||
echo "127.0.0.1 mosaic.local api.mosaic.local auth.mosaic.local" | sudo tee -a /etc/hosts
|
||||
```
|
||||
|
||||
Browser will show certificate warnings (expected for self-signed certs).
|
||||
|
||||
### Production with Let's Encrypt
|
||||
|
||||
```bash
|
||||
# .env configuration
|
||||
TRAEFIK_MODE=bundled
|
||||
TRAEFIK_ENABLE=true
|
||||
TRAEFIK_TLS_ENABLED=true
|
||||
TRAEFIK_ACME_EMAIL=admin@example.com
|
||||
TRAEFIK_CERTRESOLVER=letsencrypt
|
||||
MOSAIC_API_DOMAIN=api.example.com
|
||||
MOSAIC_WEB_DOMAIN=example.com
|
||||
MOSAIC_AUTH_DOMAIN=auth.example.com
|
||||
```
|
||||
|
||||
**Prerequisites:**
|
||||
1. DNS records pointing to your server
|
||||
2. Ports 80 and 443 accessible from internet
|
||||
3. Uncomment ACME configuration in `docker/traefik/traefik.yml`
|
||||
|
||||
```yaml
|
||||
# docker/traefik/traefik.yml
|
||||
certificatesResolvers:
|
||||
letsencrypt:
|
||||
acme:
|
||||
email: "${TRAEFIK_ACME_EMAIL}"
|
||||
storage: "/letsencrypt/acme.json"
|
||||
httpChallenge:
|
||||
entryPoint: web
|
||||
```
|
||||
|
||||
### Connecting to Existing Traefik (web1.corp.uscllc.local)
|
||||
|
||||
For the shared development environment at `~/src/traefik`:
|
||||
|
||||
```bash
|
||||
# 1. Verify external Traefik is running
|
||||
docker ps | grep traefik
|
||||
|
||||
# 2. Verify external network exists
|
||||
docker network ls | grep traefik-public
|
||||
|
||||
# 3. Configure .env
|
||||
TRAEFIK_MODE=upstream
|
||||
TRAEFIK_ENABLE=true
|
||||
TRAEFIK_NETWORK=traefik-public
|
||||
TRAEFIK_DOCKER_NETWORK=traefik-public
|
||||
MOSAIC_API_DOMAIN=mosaic-api.uscllc.com
|
||||
MOSAIC_WEB_DOMAIN=mosaic.uscllc.com
|
||||
MOSAIC_AUTH_DOMAIN=mosaic-auth.uscllc.com
|
||||
|
||||
# 4. Create docker-compose.override.yml
|
||||
cp docker-compose.override.yml.example docker-compose.override.yml
|
||||
|
||||
# 5. Edit docker-compose.override.yml - uncomment network section
|
||||
# networks:
|
||||
# traefik-public:
|
||||
# external: true
|
||||
# name: traefik-public
|
||||
|
||||
# 6. Add services to external network
|
||||
# api:
|
||||
# networks:
|
||||
# - traefik-public
|
||||
# web:
|
||||
# networks:
|
||||
# - traefik-public
|
||||
# authentik-server:
|
||||
# networks:
|
||||
# - traefik-public
|
||||
|
||||
# 7. Start services
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Services will be auto-discovered by the external Traefik instance.
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Custom Middleware
|
||||
|
||||
Add authentication or rate limiting via Traefik middleware:
|
||||
|
||||
```yaml
|
||||
# docker-compose.override.yml
|
||||
services:
|
||||
traefik:
|
||||
labels:
|
||||
# Define basic auth middleware
|
||||
- "traefik.http.middlewares.auth.basicauth.users=admin:$$apr1$$xyz..."
|
||||
# Define rate limit middleware
|
||||
- "traefik.http.middlewares.ratelimit.ratelimit.average=100"
|
||||
- "traefik.http.middlewares.ratelimit.ratelimit.burst=50"
|
||||
|
||||
api:
|
||||
labels:
|
||||
# Apply middleware to API router
|
||||
- "traefik.http.routers.mosaic-api.middlewares=auth@docker,ratelimit@docker"
|
||||
```
|
||||
|
||||
Generate basic auth password:
|
||||
```bash
|
||||
echo $(htpasswd -nb admin your-password) | sed -e s/\\$/\\$\\$/g
|
||||
```
|
||||
|
||||
### Custom TLS Certificates
|
||||
|
||||
To use custom certificates instead of Let's Encrypt:
|
||||
|
||||
```yaml
|
||||
# docker/traefik/dynamic/tls.yml
|
||||
tls:
|
||||
certificates:
|
||||
- certFile: /certs/domain.crt
|
||||
keyFile: /certs/domain.key
|
||||
```
|
||||
|
||||
Mount certificate directory:
|
||||
```yaml
|
||||
# docker-compose.override.yml
|
||||
services:
|
||||
traefik:
|
||||
volumes:
|
||||
- ./certs:/certs:ro
|
||||
```
|
||||
|
||||
### Multiple Domains
|
||||
|
||||
Route multiple domains to different services:
|
||||
|
||||
```yaml
|
||||
# .env
|
||||
MOSAIC_WEB_DOMAIN=mosaic.local,app.mosaic.local,www.mosaic.local
|
||||
|
||||
# Traefik will match all domains
|
||||
```
|
||||
|
||||
Or use override for complex routing:
|
||||
|
||||
```yaml
|
||||
# docker-compose.override.yml
|
||||
services:
|
||||
web:
|
||||
labels:
|
||||
- "traefik.http.routers.mosaic-web.rule=Host(`mosaic.local`) || Host(`app.mosaic.local`)"
|
||||
- "traefik.http.routers.mosaic-web-www.rule=Host(`www.mosaic.local`)"
|
||||
- "traefik.http.routers.mosaic-web-www.middlewares=redirect-www"
|
||||
- "traefik.http.middlewares.redirect-www.redirectregex.regex=^https://www.mosaic.local/(.*)"
|
||||
- "traefik.http.middlewares.redirect-www.redirectregex.replacement=https://mosaic.local/$${1}"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Services Not Accessible via Domain
|
||||
|
||||
**Check Traefik is running:**
|
||||
```bash
|
||||
docker ps | grep traefik
|
||||
```
|
||||
|
||||
**Check Traefik dashboard:**
|
||||
```bash
|
||||
# Bundled mode
|
||||
open http://localhost:8080
|
||||
|
||||
# Check registered routers
|
||||
curl http://localhost:8080/api/http/routers | jq
|
||||
```
|
||||
|
||||
**Verify labels are applied:**
|
||||
```bash
|
||||
docker inspect mosaic-api | jq '.Config.Labels'
|
||||
```
|
||||
|
||||
**Check DNS/hosts file:**
|
||||
```bash
|
||||
# Local development
|
||||
cat /etc/hosts | grep mosaic
|
||||
```
|
||||
|
||||
### Certificate Errors
|
||||
|
||||
**Self-signed certificates (development):**
|
||||
- Browser warnings are expected
|
||||
- Add exception in browser or import CA certificate
|
||||
|
||||
**Let's Encrypt failures:**
|
||||
```bash
|
||||
# Check Traefik logs
|
||||
docker logs mosaic-traefik
|
||||
|
||||
# Verify ACME email is set
|
||||
docker exec mosaic-traefik cat /etc/traefik/traefik.yml
|
||||
|
||||
# Check certificate storage
|
||||
docker exec mosaic-traefik ls -la /letsencrypt/
|
||||
```
|
||||
|
||||
### Upstream Mode Not Connecting
|
||||
|
||||
**Verify external network exists:**
|
||||
```bash
|
||||
docker network ls | grep traefik-public
|
||||
```
|
||||
|
||||
**Create network if missing:**
|
||||
```bash
|
||||
docker network create traefik-public
|
||||
```
|
||||
|
||||
**Check service network attachment:**
|
||||
```bash
|
||||
docker inspect mosaic-api | jq '.NetworkSettings.Networks'
|
||||
```
|
||||
|
||||
**Verify external Traefik can see services:**
|
||||
```bash
|
||||
# From external Traefik container
|
||||
docker exec <external-traefik-container> traefik healthcheck
|
||||
```
|
||||
|
||||
### Port Conflicts
|
||||
|
||||
**Bundled mode port conflicts:**
|
||||
```bash
|
||||
# Check what's using ports
|
||||
sudo lsof -i :80
|
||||
sudo lsof -i :443
|
||||
sudo lsof -i :8080
|
||||
|
||||
# Change ports in .env
|
||||
TRAEFIK_HTTP_PORT=8000
|
||||
TRAEFIK_HTTPS_PORT=8443
|
||||
TRAEFIK_DASHBOARD_PORT=8081
|
||||
```
|
||||
|
||||
### Dashboard Not Accessible
|
||||
|
||||
**Check dashboard is enabled:**
|
||||
```bash
|
||||
# In .env
|
||||
TRAEFIK_DASHBOARD_ENABLED=true
|
||||
```
|
||||
|
||||
**Verify Traefik configuration:**
|
||||
```bash
|
||||
docker exec mosaic-traefik cat /etc/traefik/traefik.yml | grep -A5 "api:"
|
||||
```
|
||||
|
||||
**Access dashboard:**
|
||||
```bash
|
||||
# Default
|
||||
http://localhost:8080/dashboard/
|
||||
|
||||
# Custom port
|
||||
http://localhost:${TRAEFIK_DASHBOARD_PORT}/dashboard/
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Production Checklist
|
||||
|
||||
- [ ] Use Let's Encrypt or valid SSL certificates
|
||||
- [ ] Disable Traefik dashboard or protect with authentication
|
||||
- [ ] Enable HTTP to HTTPS redirect
|
||||
- [ ] Configure rate limiting middleware
|
||||
- [ ] Use strong passwords for all services
|
||||
- [ ] Restrict Traefik dashboard to internal network
|
||||
- [ ] Enable Traefik access logs for audit trail
|
||||
- [ ] Regularly update Traefik image version
|
||||
|
||||
### Securing the Dashboard
|
||||
|
||||
**Option 1: Disable in production**
|
||||
```bash
|
||||
TRAEFIK_DASHBOARD_ENABLED=false
|
||||
```
|
||||
|
||||
**Option 2: Add basic authentication**
|
||||
```yaml
|
||||
# docker-compose.override.yml
|
||||
services:
|
||||
traefik:
|
||||
command:
|
||||
- "--configFile=/etc/traefik/traefik.yml"
|
||||
- "--api.dashboard=true"
|
||||
labels:
|
||||
- "traefik.http.routers.dashboard.rule=Host(`traefik.example.com`)"
|
||||
- "traefik.http.routers.dashboard.service=api@internal"
|
||||
- "traefik.http.routers.dashboard.middlewares=auth"
|
||||
- "traefik.http.middlewares.auth.basicauth.users=admin:$$apr1$$xyz..."
|
||||
```
|
||||
|
||||
**Option 3: IP whitelist**
|
||||
```yaml
|
||||
# docker-compose.override.yml
|
||||
services:
|
||||
traefik:
|
||||
labels:
|
||||
- "traefik.http.middlewares.ipwhitelist.ipwhitelist.sourcerange=127.0.0.1/32,10.0.0.0/8"
|
||||
- "traefik.http.routers.dashboard.middlewares=ipwhitelist"
|
||||
```
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
### Connection Limits
|
||||
|
||||
```yaml
|
||||
# docker/traefik/traefik.yml
|
||||
entryPoints:
|
||||
web:
|
||||
address: ":80"
|
||||
transport:
|
||||
respondingTimeouts:
|
||||
readTimeout: 60
|
||||
writeTimeout: 60
|
||||
idleTimeout: 180
|
||||
websecure:
|
||||
address: ":443"
|
||||
transport:
|
||||
respondingTimeouts:
|
||||
readTimeout: 60
|
||||
writeTimeout: 60
|
||||
idleTimeout: 180
|
||||
```
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
```yaml
|
||||
# docker-compose.override.yml
|
||||
services:
|
||||
traefik:
|
||||
labels:
|
||||
- "traefik.http.middlewares.ratelimit.ratelimit.average=100"
|
||||
- "traefik.http.middlewares.ratelimit.ratelimit.burst=200"
|
||||
- "traefik.http.middlewares.ratelimit.ratelimit.period=1s"
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
Integration tests are available to verify Traefik configuration:
|
||||
|
||||
```bash
|
||||
# Run all Traefik tests
|
||||
./tests/integration/docker/traefik.test.sh all
|
||||
|
||||
# Test specific mode
|
||||
./tests/integration/docker/traefik.test.sh bundled
|
||||
./tests/integration/docker/traefik.test.sh upstream
|
||||
./tests/integration/docker/traefik.test.sh none
|
||||
```
|
||||
|
||||
See `tests/integration/docker/README.md` for details.
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### From None Mode to Bundled Mode
|
||||
|
||||
```bash
|
||||
# 1. Stop existing services
|
||||
docker compose down
|
||||
|
||||
# 2. Backup current .env
|
||||
cp .env .env.backup
|
||||
|
||||
# 3. Switch to bundled configuration
|
||||
cp .env.traefik-bundled.example .env
|
||||
|
||||
# 4. Transfer existing secrets from .env.backup to .env
|
||||
|
||||
# 5. Start with Traefik profile
|
||||
docker compose --profile traefik-bundled up -d
|
||||
|
||||
# 6. Update application URLs if needed
|
||||
# Old: http://localhost:3000
|
||||
# New: https://mosaic.local
|
||||
```
|
||||
|
||||
### From Bundled Mode to Upstream Mode
|
||||
|
||||
```bash
|
||||
# 1. Ensure external Traefik is running
|
||||
docker network create traefik-public
|
||||
|
||||
# 2. Update .env
|
||||
TRAEFIK_MODE=upstream
|
||||
TRAEFIK_NETWORK=traefik-public
|
||||
|
||||
# 3. Create override file
|
||||
cp docker-compose.override.yml.example docker-compose.override.yml
|
||||
|
||||
# 4. Edit override file to uncomment network section
|
||||
|
||||
# 5. Restart without bundled profile
|
||||
docker compose down
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [Traefik Official Documentation](https://doc.traefik.io/traefik/)
|
||||
- [Traefik Docker Provider](https://doc.traefik.io/traefik/providers/docker/)
|
||||
- [Let's Encrypt with Traefik](https://doc.traefik.io/traefik/https/acme/)
|
||||
- [Traefik Middleware Reference](https://doc.traefik.io/traefik/middlewares/overview/)
|
||||
Reference in New Issue
Block a user