Implements FED-010: Agent Spawn via Federation feature that enables spawning and managing Claude agents on remote federated Mosaic Stack instances via COMMAND message type. Features: - Federation agent command types (spawn, status, kill) - FederationAgentService for handling agent operations - Integration with orchestrator's agent spawner/lifecycle services - API endpoints for spawning, querying status, and killing agents - Full command routing through federation COMMAND infrastructure - Comprehensive test coverage (12/12 tests passing) Architecture: - Hub → Spoke: Spawn agents on remote instances - Command flow: FederationController → FederationAgentService → CommandService → Remote Orchestrator - Response handling: Remote orchestrator returns agent status/results - Security: Connection validation, signature verification Files created: - apps/api/src/federation/types/federation-agent.types.ts - apps/api/src/federation/federation-agent.service.ts - apps/api/src/federation/federation-agent.service.spec.ts Files modified: - apps/api/src/federation/command.service.ts (agent command routing) - apps/api/src/federation/federation.controller.ts (agent endpoints) - apps/api/src/federation/federation.module.ts (service registration) - apps/orchestrator/src/api/agents/agents.controller.ts (status endpoint) - apps/orchestrator/src/api/agents/agents.module.ts (lifecycle integration) Testing: - 12/12 tests passing for FederationAgentService - All command service tests passing - TypeScript compilation successful - Linting passed Refs #93 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
447 lines
8.4 KiB
Markdown
447 lines
8.4 KiB
Markdown
# Docker Configuration
|
|
|
|
Configuration guide specific to Docker Compose deployments.
|
|
|
|
## Overview
|
|
|
|
Docker Compose deployments use environment variables to configure all services. This guide covers Docker-specific configuration options.
|
|
|
|
## Environment File
|
|
|
|
All Docker configurations are in `.env` at the project root:
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
nano .env
|
|
```
|
|
|
|
## Service Configuration
|
|
|
|
### Application Ports
|
|
|
|
```bash
|
|
# API port (external mapping)
|
|
API_PORT=3001
|
|
API_HOST=0.0.0.0
|
|
|
|
# Web port (external mapping)
|
|
WEB_PORT=3000
|
|
|
|
# Public API URL (for Next.js client)
|
|
NEXT_PUBLIC_API_URL=http://localhost:3001
|
|
```
|
|
|
|
### PostgreSQL Database
|
|
|
|
```bash
|
|
# Connection string for API (uses Docker internal networking)
|
|
DATABASE_URL=postgresql://mosaic:mosaic_dev_password@postgres:5432/mosaic
|
|
|
|
# PostgreSQL container configuration
|
|
POSTGRES_USER=mosaic
|
|
POSTGRES_PASSWORD=mosaic_dev_password
|
|
POSTGRES_DB=mosaic
|
|
POSTGRES_PORT=5432
|
|
|
|
# Performance tuning (optional)
|
|
POSTGRES_SHARED_BUFFERS=256MB
|
|
POSTGRES_EFFECTIVE_CACHE_SIZE=1GB
|
|
POSTGRES_MAX_CONNECTIONS=100
|
|
```
|
|
|
|
**Important:** For Docker deployments, use `postgres` as the hostname (container name), not `localhost`.
|
|
|
|
### Valkey Cache
|
|
|
|
```bash
|
|
# Connection string for API (uses Docker internal networking)
|
|
VALKEY_URL=redis://valkey:6379
|
|
|
|
# Valkey container configuration
|
|
VALKEY_PORT=6379
|
|
VALKEY_MAXMEMORY=256mb
|
|
```
|
|
|
|
**Important:** For Docker deployments, use `valkey` as the hostname (container name), not `localhost`.
|
|
|
|
### Authentik OIDC (Optional)
|
|
|
|
When using the bundled Authentik service:
|
|
|
|
```bash
|
|
# Authentik PostgreSQL
|
|
AUTHENTIK_POSTGRES_USER=authentik
|
|
AUTHENTIK_POSTGRES_PASSWORD=authentik_password
|
|
AUTHENTIK_POSTGRES_DB=authentik
|
|
|
|
# Authentik Server Configuration
|
|
AUTHENTIK_SECRET_KEY=change-this-to-a-random-secret-key-minimum-50-characters
|
|
AUTHENTIK_ERROR_REPORTING=false
|
|
AUTHENTIK_BOOTSTRAP_PASSWORD=admin
|
|
AUTHENTIK_BOOTSTRAP_EMAIL=admin@localhost
|
|
AUTHENTIK_COOKIE_DOMAIN=.localhost
|
|
|
|
# Authentik Ports
|
|
AUTHENTIK_PORT_HTTP=9000
|
|
AUTHENTIK_PORT_HTTPS=9443
|
|
|
|
# OIDC Configuration (configured in Authentik UI)
|
|
OIDC_ISSUER=http://localhost:9000/application/o/mosaic-stack/
|
|
OIDC_CLIENT_ID=your-client-id-here
|
|
OIDC_CLIENT_SECRET=your-client-secret-here
|
|
OIDC_REDIRECT_URI=http://localhost:3001/auth/callback
|
|
```
|
|
|
|
**Bootstrap Credentials:**
|
|
|
|
- Username: `akadmin`
|
|
- Password: Value of `AUTHENTIK_BOOTSTRAP_PASSWORD`
|
|
|
|
### Ollama AI Service (Optional)
|
|
|
|
When using the bundled Ollama service:
|
|
|
|
```bash
|
|
# Ollama endpoint (uses Docker internal networking)
|
|
OLLAMA_ENDPOINT=http://ollama:11434
|
|
|
|
# Ollama port (external mapping)
|
|
OLLAMA_PORT=11434
|
|
```
|
|
|
|
## Docker Compose Profiles
|
|
|
|
Control which optional services are started using profiles:
|
|
|
|
```bash
|
|
# Option 1: Command line
|
|
docker compose --profile authentik up -d
|
|
docker compose --profile ollama up -d
|
|
docker compose --profile full up -d
|
|
|
|
# Option 2: Environment variable
|
|
COMPOSE_PROFILES=authentik,ollama # Enable specific services
|
|
COMPOSE_PROFILES=full # Enable all optional services
|
|
```
|
|
|
|
Available profiles:
|
|
|
|
- `authentik` - Authentik OIDC provider stack
|
|
- `ollama` - Ollama LLM service
|
|
- `full` - All optional services
|
|
|
|
## Security Configuration
|
|
|
|
### Production Secrets
|
|
|
|
**CRITICAL:** Change these in production:
|
|
|
|
```bash
|
|
# PostgreSQL
|
|
POSTGRES_PASSWORD=$(openssl rand -base64 32)
|
|
|
|
# Authentik
|
|
AUTHENTIK_SECRET_KEY=$(openssl rand -base64 50)
|
|
AUTHENTIK_POSTGRES_PASSWORD=$(openssl rand -base64 32)
|
|
AUTHENTIK_BOOTSTRAP_PASSWORD=$(openssl rand -base64 16)
|
|
|
|
# JWT
|
|
JWT_SECRET=$(openssl rand -base64 32)
|
|
```
|
|
|
|
### Network Security
|
|
|
|
The Docker setup uses two networks:
|
|
|
|
1. **mosaic-internal** (internal only)
|
|
- PostgreSQL
|
|
- Valkey
|
|
- Authentik PostgreSQL
|
|
- Authentik Redis
|
|
- No external access
|
|
|
|
2. **mosaic-public** (external access)
|
|
- API
|
|
- Web
|
|
- Authentik Server
|
|
- Accessible from host
|
|
|
|
## Volume Management
|
|
|
|
### Persistent Volumes
|
|
|
|
Data is stored in named Docker volumes:
|
|
|
|
```bash
|
|
# List volumes
|
|
docker volume ls | grep mosaic
|
|
|
|
# Inspect volume
|
|
docker volume inspect mosaic-postgres-data
|
|
|
|
# Backup volume
|
|
docker run --rm \
|
|
-v mosaic-postgres-data:/data \
|
|
-v $(pwd):/backup \
|
|
alpine tar czf /backup/postgres-backup.tar.gz /data
|
|
|
|
# Restore volume
|
|
docker run --rm \
|
|
-v mosaic-postgres-data:/data \
|
|
-v $(pwd):/backup \
|
|
alpine tar xzf /backup/postgres-backup.tar.gz -C /
|
|
```
|
|
|
|
### Volume Locations
|
|
|
|
- `mosaic-postgres-data` - PostgreSQL database files
|
|
- `mosaic-valkey-data` - Valkey persistence
|
|
- `mosaic-authentik-postgres-data` - Authentik database
|
|
- `mosaic-authentik-redis-data` - Authentik cache
|
|
- `mosaic-authentik-media` - Authentik uploaded files
|
|
- `mosaic-authentik-certs` - Authentik certificates
|
|
- `mosaic-authentik-templates` - Authentik email templates
|
|
- `mosaic-ollama-data` - Ollama models
|
|
|
|
## Custom Configurations
|
|
|
|
### Using External Services
|
|
|
|
Create `docker-compose.override.yml` to use external services:
|
|
|
|
```yaml
|
|
# Disable bundled PostgreSQL, use external
|
|
services:
|
|
postgres:
|
|
profiles:
|
|
- disabled
|
|
|
|
api:
|
|
environment:
|
|
DATABASE_URL: postgresql://user:pass@external-db.example.com:5432/mosaic
|
|
```
|
|
|
|
See `docker-compose.override.yml.example` for more examples.
|
|
|
|
### Development Overrides
|
|
|
|
```yaml
|
|
# docker-compose.override.yml
|
|
services:
|
|
postgres:
|
|
command:
|
|
- "postgres"
|
|
- "-c"
|
|
- "log_statement=all"
|
|
- "-c"
|
|
- "log_duration=on"
|
|
ports:
|
|
- "5432:5432"
|
|
|
|
api:
|
|
environment:
|
|
LOG_LEVEL: debug
|
|
volumes:
|
|
- ./apps/api/src:/app/apps/api/src:ro
|
|
```
|
|
|
|
### Production Overrides
|
|
|
|
```yaml
|
|
# docker-compose.prod.yml
|
|
services:
|
|
api:
|
|
restart: always
|
|
environment:
|
|
NODE_ENV: production
|
|
LOG_LEVEL: warn
|
|
deploy:
|
|
replicas: 2
|
|
resources:
|
|
limits:
|
|
cpus: "1.0"
|
|
memory: 1G
|
|
|
|
web:
|
|
restart: always
|
|
environment:
|
|
NODE_ENV: production
|
|
deploy:
|
|
replicas: 2
|
|
resources:
|
|
limits:
|
|
cpus: "0.5"
|
|
memory: 512M
|
|
```
|
|
|
|
Deploy:
|
|
|
|
```bash
|
|
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
|
```
|
|
|
|
## Resource Limits
|
|
|
|
### Memory Limits
|
|
|
|
Adjust based on your system:
|
|
|
|
```yaml
|
|
# docker-compose.override.yml
|
|
services:
|
|
postgres:
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
memory: 2G
|
|
reservations:
|
|
memory: 512M
|
|
|
|
api:
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
memory: 1G
|
|
reservations:
|
|
memory: 256M
|
|
```
|
|
|
|
### CPU Limits
|
|
|
|
```yaml
|
|
services:
|
|
api:
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
cpus: "1.0"
|
|
reservations:
|
|
cpus: "0.25"
|
|
```
|
|
|
|
## Health Checks
|
|
|
|
All services include health checks. Adjust timing if needed:
|
|
|
|
```yaml
|
|
# docker-compose.override.yml
|
|
services:
|
|
postgres:
|
|
healthcheck:
|
|
interval: 30s # Check every 30s
|
|
timeout: 10s # Timeout after 10s
|
|
retries: 5 # Retry 5 times
|
|
start_period: 60s # Wait 60s before first check
|
|
```
|
|
|
|
## Logging Configuration
|
|
|
|
### Log Drivers
|
|
|
|
```yaml
|
|
# docker-compose.override.yml
|
|
services:
|
|
api:
|
|
logging:
|
|
driver: "json-file"
|
|
options:
|
|
max-size: "10m"
|
|
max-file: "3"
|
|
```
|
|
|
|
### Centralized Logging
|
|
|
|
For production, consider:
|
|
|
|
- Loki + Grafana
|
|
- ELK Stack (Elasticsearch, Logstash, Kibana)
|
|
- Fluentd
|
|
- CloudWatch Logs
|
|
|
|
Example with Loki:
|
|
|
|
```yaml
|
|
services:
|
|
api:
|
|
logging:
|
|
driver: loki
|
|
options:
|
|
loki-url: "http://loki:3100/loki/api/v1/push"
|
|
labels: "service=mosaic-api"
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Container Won't Start
|
|
|
|
Check logs:
|
|
|
|
```bash
|
|
docker compose logs <service>
|
|
```
|
|
|
|
Common issues:
|
|
|
|
- Port conflict: Change port in `.env`
|
|
- Missing environment variable: Check `.env` file
|
|
- Health check failing: Increase `start_period`
|
|
|
|
### Network Issues
|
|
|
|
Test connectivity between containers:
|
|
|
|
```bash
|
|
# From API container to PostgreSQL
|
|
docker compose exec api sh
|
|
nc -zv postgres 5432
|
|
```
|
|
|
|
### Volume Permission Issues
|
|
|
|
Fix permissions:
|
|
|
|
```bash
|
|
# PostgreSQL volume
|
|
docker compose exec postgres chown -R postgres:postgres /var/lib/postgresql/data
|
|
```
|
|
|
|
### Out of Disk Space
|
|
|
|
Clean up:
|
|
|
|
```bash
|
|
# Remove unused containers, networks, images
|
|
docker system prune -a
|
|
|
|
# Remove unused volumes (WARNING: deletes data)
|
|
docker volume prune
|
|
```
|
|
|
|
## Monitoring
|
|
|
|
### Resource Usage
|
|
|
|
```bash
|
|
# Real-time stats
|
|
docker stats
|
|
|
|
# Specific container
|
|
docker stats mosaic-api
|
|
```
|
|
|
|
### Health Status
|
|
|
|
```bash
|
|
# Check health of all services
|
|
docker compose ps
|
|
|
|
# JSON output
|
|
docker compose ps --format json
|
|
```
|
|
|
|
## Next Steps
|
|
|
|
- [Docker Deployment Guide](../4-docker-deployment/README.md) - Complete deployment guide
|
|
- [Authentik Setup](2-authentik.md) - Configure OIDC authentication
|
|
- [Environment Variables](1-environment.md) - Full variable reference
|