feat(#37-41): Add domains, ideas, relationships, agents, widgets schema
Schema additions for issues #37-41: New models: - Domain (#37): Life domains (work, marriage, homelab, etc.) - Idea (#38): Brain dumps with pgvector embeddings - Relationship (#39): Generic entity linking (blocks, depends_on) - Agent (#40): ClawdBot agent tracking with metrics - AgentSession (#40): Conversation session tracking - WidgetDefinition (#41): HUD widget registry - UserLayout (#41): Per-user dashboard configuration Updated models: - Task, Event, Project: Added domainId foreign key - User, Workspace: Added new relations New enums: - IdeaStatus: CAPTURED, PROCESSING, ACTIONABLE, ARCHIVED, DISCARDED - RelationshipType: BLOCKS, BLOCKED_BY, DEPENDS_ON, etc. - AgentStatus: IDLE, WORKING, WAITING, ERROR, TERMINATED - EntityType: Added IDEA, DOMAIN Migration: 20260129182803_add_domains_ideas_agents_widgets
This commit is contained in:
Jason Woltje
437
docs/1-getting-started/3-configuration/3-docker.md
Normal file
437
docs/1-getting-started/3-configuration/3-docker.md
Normal file
@@ -0,0 +1,437 @@
|
||||
# Docker Configuration
|
||||
|
||||
Configuration guide specific to Docker Compose deployments.
|
||||
|
||||
## Overview
|
||||
|
||||
Docker Compose deployments use environment variables to configure all services. This guide covers Docker-specific configuration options.
|
||||
|
||||
## Environment File
|
||||
|
||||
All Docker configurations are in `.env` at the project root:
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
nano .env
|
||||
```
|
||||
|
||||
## Service Configuration
|
||||
|
||||
### Application Ports
|
||||
|
||||
```bash
|
||||
# API port (external mapping)
|
||||
API_PORT=3001
|
||||
API_HOST=0.0.0.0
|
||||
|
||||
# Web port (external mapping)
|
||||
WEB_PORT=3000
|
||||
|
||||
# Public API URL (for Next.js client)
|
||||
NEXT_PUBLIC_API_URL=http://localhost:3001
|
||||
```
|
||||
|
||||
### PostgreSQL Database
|
||||
|
||||
```bash
|
||||
# Connection string for API (uses Docker internal networking)
|
||||
DATABASE_URL=postgresql://mosaic:mosaic_dev_password@postgres:5432/mosaic
|
||||
|
||||
# PostgreSQL container configuration
|
||||
POSTGRES_USER=mosaic
|
||||
POSTGRES_PASSWORD=mosaic_dev_password
|
||||
POSTGRES_DB=mosaic
|
||||
POSTGRES_PORT=5432
|
||||
|
||||
# Performance tuning (optional)
|
||||
POSTGRES_SHARED_BUFFERS=256MB
|
||||
POSTGRES_EFFECTIVE_CACHE_SIZE=1GB
|
||||
POSTGRES_MAX_CONNECTIONS=100
|
||||
```
|
||||
|
||||
**Important:** For Docker deployments, use `postgres` as the hostname (container name), not `localhost`.
|
||||
|
||||
### Valkey Cache
|
||||
|
||||
```bash
|
||||
# Connection string for API (uses Docker internal networking)
|
||||
VALKEY_URL=redis://valkey:6379
|
||||
|
||||
# Valkey container configuration
|
||||
VALKEY_PORT=6379
|
||||
VALKEY_MAXMEMORY=256mb
|
||||
```
|
||||
|
||||
**Important:** For Docker deployments, use `valkey` as the hostname (container name), not `localhost`.
|
||||
|
||||
### Authentik OIDC (Optional)
|
||||
|
||||
When using the bundled Authentik service:
|
||||
|
||||
```bash
|
||||
# Authentik PostgreSQL
|
||||
AUTHENTIK_POSTGRES_USER=authentik
|
||||
AUTHENTIK_POSTGRES_PASSWORD=authentik_password
|
||||
AUTHENTIK_POSTGRES_DB=authentik
|
||||
|
||||
# Authentik Server Configuration
|
||||
AUTHENTIK_SECRET_KEY=change-this-to-a-random-secret-key-minimum-50-characters
|
||||
AUTHENTIK_ERROR_REPORTING=false
|
||||
AUTHENTIK_BOOTSTRAP_PASSWORD=admin
|
||||
AUTHENTIK_BOOTSTRAP_EMAIL=admin@localhost
|
||||
AUTHENTIK_COOKIE_DOMAIN=.localhost
|
||||
|
||||
# Authentik Ports
|
||||
AUTHENTIK_PORT_HTTP=9000
|
||||
AUTHENTIK_PORT_HTTPS=9443
|
||||
|
||||
# OIDC Configuration (configured in Authentik UI)
|
||||
OIDC_ISSUER=http://localhost:9000/application/o/mosaic-stack/
|
||||
OIDC_CLIENT_ID=your-client-id-here
|
||||
OIDC_CLIENT_SECRET=your-client-secret-here
|
||||
OIDC_REDIRECT_URI=http://localhost:3001/auth/callback
|
||||
```
|
||||
|
||||
**Bootstrap Credentials:**
|
||||
- Username: `akadmin`
|
||||
- Password: Value of `AUTHENTIK_BOOTSTRAP_PASSWORD`
|
||||
|
||||
### Ollama AI Service (Optional)
|
||||
|
||||
When using the bundled Ollama service:
|
||||
|
||||
```bash
|
||||
# Ollama endpoint (uses Docker internal networking)
|
||||
OLLAMA_ENDPOINT=http://ollama:11434
|
||||
|
||||
# Ollama port (external mapping)
|
||||
OLLAMA_PORT=11434
|
||||
```
|
||||
|
||||
## Docker Compose Profiles
|
||||
|
||||
Control which optional services are started using profiles:
|
||||
|
||||
```bash
|
||||
# Option 1: Command line
|
||||
docker compose --profile authentik up -d
|
||||
docker compose --profile ollama up -d
|
||||
docker compose --profile full up -d
|
||||
|
||||
# Option 2: Environment variable
|
||||
COMPOSE_PROFILES=authentik,ollama # Enable specific services
|
||||
COMPOSE_PROFILES=full # Enable all optional services
|
||||
```
|
||||
|
||||
Available profiles:
|
||||
- `authentik` - Authentik OIDC provider stack
|
||||
- `ollama` - Ollama LLM service
|
||||
- `full` - All optional services
|
||||
|
||||
## Security Configuration
|
||||
|
||||
### Production Secrets
|
||||
|
||||
**CRITICAL:** Change these in production:
|
||||
|
||||
```bash
|
||||
# PostgreSQL
|
||||
POSTGRES_PASSWORD=$(openssl rand -base64 32)
|
||||
|
||||
# Authentik
|
||||
AUTHENTIK_SECRET_KEY=$(openssl rand -base64 50)
|
||||
AUTHENTIK_POSTGRES_PASSWORD=$(openssl rand -base64 32)
|
||||
AUTHENTIK_BOOTSTRAP_PASSWORD=$(openssl rand -base64 16)
|
||||
|
||||
# JWT
|
||||
JWT_SECRET=$(openssl rand -base64 32)
|
||||
```
|
||||
|
||||
### Network Security
|
||||
|
||||
The Docker setup uses two networks:
|
||||
|
||||
1. **mosaic-internal** (internal only)
|
||||
- PostgreSQL
|
||||
- Valkey
|
||||
- Authentik PostgreSQL
|
||||
- Authentik Redis
|
||||
- No external access
|
||||
|
||||
2. **mosaic-public** (external access)
|
||||
- API
|
||||
- Web
|
||||
- Authentik Server
|
||||
- Accessible from host
|
||||
|
||||
## Volume Management
|
||||
|
||||
### Persistent Volumes
|
||||
|
||||
Data is stored in named Docker volumes:
|
||||
|
||||
```bash
|
||||
# List volumes
|
||||
docker volume ls | grep mosaic
|
||||
|
||||
# Inspect volume
|
||||
docker volume inspect mosaic-postgres-data
|
||||
|
||||
# Backup volume
|
||||
docker run --rm \
|
||||
-v mosaic-postgres-data:/data \
|
||||
-v $(pwd):/backup \
|
||||
alpine tar czf /backup/postgres-backup.tar.gz /data
|
||||
|
||||
# Restore volume
|
||||
docker run --rm \
|
||||
-v mosaic-postgres-data:/data \
|
||||
-v $(pwd):/backup \
|
||||
alpine tar xzf /backup/postgres-backup.tar.gz -C /
|
||||
```
|
||||
|
||||
### Volume Locations
|
||||
|
||||
- `mosaic-postgres-data` - PostgreSQL database files
|
||||
- `mosaic-valkey-data` - Valkey persistence
|
||||
- `mosaic-authentik-postgres-data` - Authentik database
|
||||
- `mosaic-authentik-redis-data` - Authentik cache
|
||||
- `mosaic-authentik-media` - Authentik uploaded files
|
||||
- `mosaic-authentik-certs` - Authentik certificates
|
||||
- `mosaic-authentik-templates` - Authentik email templates
|
||||
- `mosaic-ollama-data` - Ollama models
|
||||
|
||||
## Custom Configurations
|
||||
|
||||
### Using External Services
|
||||
|
||||
Create `docker-compose.override.yml` to use external services:
|
||||
|
||||
```yaml
|
||||
# Disable bundled PostgreSQL, use external
|
||||
services:
|
||||
postgres:
|
||||
profiles:
|
||||
- disabled
|
||||
|
||||
api:
|
||||
environment:
|
||||
DATABASE_URL: postgresql://user:pass@external-db.example.com:5432/mosaic
|
||||
```
|
||||
|
||||
See `docker-compose.override.yml.example` for more examples.
|
||||
|
||||
### Development Overrides
|
||||
|
||||
```yaml
|
||||
# docker-compose.override.yml
|
||||
services:
|
||||
postgres:
|
||||
command:
|
||||
- "postgres"
|
||||
- "-c"
|
||||
- "log_statement=all"
|
||||
- "-c"
|
||||
- "log_duration=on"
|
||||
ports:
|
||||
- "5432:5432"
|
||||
|
||||
api:
|
||||
environment:
|
||||
LOG_LEVEL: debug
|
||||
volumes:
|
||||
- ./apps/api/src:/app/apps/api/src:ro
|
||||
```
|
||||
|
||||
### Production Overrides
|
||||
|
||||
```yaml
|
||||
# docker-compose.prod.yml
|
||||
services:
|
||||
api:
|
||||
restart: always
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
LOG_LEVEL: warn
|
||||
deploy:
|
||||
replicas: 2
|
||||
resources:
|
||||
limits:
|
||||
cpus: '1.0'
|
||||
memory: 1G
|
||||
|
||||
web:
|
||||
restart: always
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
deploy:
|
||||
replicas: 2
|
||||
resources:
|
||||
limits:
|
||||
cpus: '0.5'
|
||||
memory: 512M
|
||||
```
|
||||
|
||||
Deploy:
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
||||
```
|
||||
|
||||
## Resource Limits
|
||||
|
||||
### Memory Limits
|
||||
|
||||
Adjust based on your system:
|
||||
|
||||
```yaml
|
||||
# docker-compose.override.yml
|
||||
services:
|
||||
postgres:
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 2G
|
||||
reservations:
|
||||
memory: 512M
|
||||
|
||||
api:
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 1G
|
||||
reservations:
|
||||
memory: 256M
|
||||
```
|
||||
|
||||
### CPU Limits
|
||||
|
||||
```yaml
|
||||
services:
|
||||
api:
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '1.0'
|
||||
reservations:
|
||||
cpus: '0.25'
|
||||
```
|
||||
|
||||
## Health Checks
|
||||
|
||||
All services include health checks. Adjust timing if needed:
|
||||
|
||||
```yaml
|
||||
# docker-compose.override.yml
|
||||
services:
|
||||
postgres:
|
||||
healthcheck:
|
||||
interval: 30s # Check every 30s
|
||||
timeout: 10s # Timeout after 10s
|
||||
retries: 5 # Retry 5 times
|
||||
start_period: 60s # Wait 60s before first check
|
||||
```
|
||||
|
||||
## Logging Configuration
|
||||
|
||||
### Log Drivers
|
||||
|
||||
```yaml
|
||||
# docker-compose.override.yml
|
||||
services:
|
||||
api:
|
||||
logging:
|
||||
driver: "json-file"
|
||||
options:
|
||||
max-size: "10m"
|
||||
max-file: "3"
|
||||
```
|
||||
|
||||
### Centralized Logging
|
||||
|
||||
For production, consider:
|
||||
- Loki + Grafana
|
||||
- ELK Stack (Elasticsearch, Logstash, Kibana)
|
||||
- Fluentd
|
||||
- CloudWatch Logs
|
||||
|
||||
Example with Loki:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
api:
|
||||
logging:
|
||||
driver: loki
|
||||
options:
|
||||
loki-url: "http://loki:3100/loki/api/v1/push"
|
||||
labels: "service=mosaic-api"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Container Won't Start
|
||||
|
||||
Check logs:
|
||||
```bash
|
||||
docker compose logs <service>
|
||||
```
|
||||
|
||||
Common issues:
|
||||
- Port conflict: Change port in `.env`
|
||||
- Missing environment variable: Check `.env` file
|
||||
- Health check failing: Increase `start_period`
|
||||
|
||||
### Network Issues
|
||||
|
||||
Test connectivity between containers:
|
||||
```bash
|
||||
# From API container to PostgreSQL
|
||||
docker compose exec api sh
|
||||
nc -zv postgres 5432
|
||||
```
|
||||
|
||||
### Volume Permission Issues
|
||||
|
||||
Fix permissions:
|
||||
```bash
|
||||
# PostgreSQL volume
|
||||
docker compose exec postgres chown -R postgres:postgres /var/lib/postgresql/data
|
||||
```
|
||||
|
||||
### Out of Disk Space
|
||||
|
||||
Clean up:
|
||||
```bash
|
||||
# Remove unused containers, networks, images
|
||||
docker system prune -a
|
||||
|
||||
# Remove unused volumes (WARNING: deletes data)
|
||||
docker volume prune
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Resource Usage
|
||||
|
||||
```bash
|
||||
# Real-time stats
|
||||
docker stats
|
||||
|
||||
# Specific container
|
||||
docker stats mosaic-api
|
||||
```
|
||||
|
||||
### Health Status
|
||||
|
||||
```bash
|
||||
# Check health of all services
|
||||
docker compose ps
|
||||
|
||||
# JSON output
|
||||
docker compose ps --format json
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Docker Deployment Guide](../4-docker-deployment/README.md) - Complete deployment guide
|
||||
- [Authentik Setup](2-authentik.md) - Configure OIDC authentication
|
||||
- [Environment Variables](1-environment.md) - Full variable reference
|
||||
Reference in New Issue
Block a user