mosaic/stack
1
0
Fork 0
Code Issues 10 Pull Requests Actions Packages Projects Releases 2 Wiki Activity

feat(#37-41): Add domains, ideas, relationships, agents, widgets schema

Browse Source 
Schema additions for issues #37-41:

New models:
- Domain (#37): Life domains (work, marriage, homelab, etc.)
- Idea (#38): Brain dumps with pgvector embeddings
- Relationship (#39): Generic entity linking (blocks, depends_on)
- Agent (#40): ClawdBot agent tracking with metrics
- AgentSession (#40): Conversation session tracking
- WidgetDefinition (#41): HUD widget registry
- UserLayout (#41): Per-user dashboard configuration

Updated models:
- Task, Event, Project: Added domainId foreign key
- User, Workspace: Added new relations

New enums:
- IdeaStatus: CAPTURED, PROCESSING, ACTIONABLE, ARCHIVED, DISCARDED
- RelationshipType: BLOCKS, BLOCKED_BY, DEPENDS_ON, etc.
- AgentStatus: IDLE, WORKING, WAITING, ERROR, TERMINATED
- EntityType: Added IDEA, DOMAIN

Migration: 20260129182803_add_domains_ideas_agents_widgets
This commit is contained in:
Jason Woltje
2026-01-29 12:29:21 -06:00
parent a220c2dc0a
commit 973502f26e
308 changed files with 18374 additions and 113 deletions
Download Patch File Download Diff File Expand all files Collapse all files

521
docs/1-getting-started/4-docker-deployment/traefik.md Normal file
View File

@@ -0,0 +1,521 @@
# Traefik Reverse Proxy Integration
Mosaic Stack supports flexible Traefik integration with three deployment modes:
1. **Bundled Mode**: Self-contained Traefik instance
2. **Upstream Mode**: Connect to external Traefik
3. **None Mode**: Direct port exposure (no reverse proxy)
## Quick Start
### Bundled Mode (Recommended for New Deployments)
```bash
# 1. Copy bundled configuration
cp .env.traefik-bundled.example .env
# 2. Update passwords and secrets in .env
# Edit POSTGRES_PASSWORD, AUTHENTIK_SECRET_KEY, JWT_SECRET, etc.
# 3. Start with bundled Traefik profile
docker compose --profile traefik-bundled up -d
# 4. Access services
# - Web: https://mosaic.local
# - API: https://api.mosaic.local
# - Auth: https://auth.mosaic.local
# - Traefik Dashboard: http://localhost:8080
```
### Upstream Mode (For Existing Traefik Instances)
```bash
# 1. Ensure external Traefik network exists
docker network create traefik-public
# 2. Copy upstream configuration
cp .env.traefik-upstream.example .env
# 3. Update .env with your domains and secrets
# 4. Create override file for network attachment
cp docker-compose.override.yml.example docker-compose.override.yml
# 5. Edit docker-compose.override.yml and uncomment upstream network section
# 6. Start services
docker compose up -d
```
### None Mode (Direct Port Access)
```bash
# 1. Use default .env.example
cp .env.example .env
# 2. Ensure TRAEFIK_MODE=none (default)
# 3. Start services
docker compose up -d
# 4. Access services on standard ports
# - Web: http://localhost:3000
# - API: http://localhost:3001
# - Auth: http://localhost:9000
```
## Configuration Reference
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `TRAEFIK_MODE` | `none` | Traefik mode: `bundled`, `upstream`, or `none` |
| `TRAEFIK_ENABLE` | `false` | Enable Traefik labels on services |
| `MOSAIC_API_DOMAIN` | `api.mosaic.local` | Domain for API service |
| `MOSAIC_WEB_DOMAIN` | `mosaic.local` | Domain for Web service |
| `MOSAIC_AUTH_DOMAIN` | `auth.mosaic.local` | Domain for Authentik service |
| `TRAEFIK_NETWORK` | `traefik-public` | External Traefik network (upstream mode) |
| `TRAEFIK_TLS_ENABLED` | `true` | Enable TLS/HTTPS |
| `TRAEFIK_ACME_EMAIL` | - | Email for Let's Encrypt (production) |
| `TRAEFIK_CERTRESOLVER` | - | Cert resolver name (e.g., `letsencrypt`) |
| `TRAEFIK_DASHBOARD_ENABLED` | `true` | Enable Traefik dashboard (bundled mode) |
| `TRAEFIK_DASHBOARD_PORT` | `8080` | Dashboard port (bundled mode) |
| `TRAEFIK_ENTRYPOINT` | `websecure` | Traefik entrypoint (`web` or `websecure`) |
| `TRAEFIK_DOCKER_NETWORK` | `mosaic-public` | Docker network for Traefik routing |
### Docker Compose Profiles
| Profile | Description |
|---------|-------------|
| `traefik-bundled` | Activates bundled Traefik service |
| `authentik` | Enables Authentik SSO services |
| `ollama` | Enables Ollama AI service |
| `full` | Enables all optional services |
## Deployment Scenarios
### Local Development with Self-Signed Certificates
```bash
# .env configuration
TRAEFIK_MODE=bundled
TRAEFIK_ENABLE=true
TRAEFIK_TLS_ENABLED=true
TRAEFIK_ACME_EMAIL= # Empty for self-signed
MOSAIC_API_DOMAIN=api.mosaic.local
MOSAIC_WEB_DOMAIN=mosaic.local
MOSAIC_AUTH_DOMAIN=auth.mosaic.local
# Start services
docker compose --profile traefik-bundled up -d
# Add to /etc/hosts
echo "127.0.0.1 mosaic.local api.mosaic.local auth.mosaic.local" | sudo tee -a /etc/hosts
```
Browser will show certificate warnings (expected for self-signed certs).
### Production with Let's Encrypt
```bash
# .env configuration
TRAEFIK_MODE=bundled
TRAEFIK_ENABLE=true
TRAEFIK_TLS_ENABLED=true
TRAEFIK_ACME_EMAIL=admin@example.com
TRAEFIK_CERTRESOLVER=letsencrypt
MOSAIC_API_DOMAIN=api.example.com
MOSAIC_WEB_DOMAIN=example.com
MOSAIC_AUTH_DOMAIN=auth.example.com
```
**Prerequisites:**
1. DNS records pointing to your server
2. Ports 80 and 443 accessible from internet
3. Uncomment ACME configuration in `docker/traefik/traefik.yml`
```yaml
# docker/traefik/traefik.yml
certificatesResolvers:
letsencrypt:
acme:
email: "${TRAEFIK_ACME_EMAIL}"
storage: "/letsencrypt/acme.json"
httpChallenge:
entryPoint: web
```
### Connecting to Existing Traefik (web1.corp.uscllc.local)
For the shared development environment at `~/src/traefik`:
```bash
# 1. Verify external Traefik is running
docker ps | grep traefik
# 2. Verify external network exists
docker network ls | grep traefik-public
# 3. Configure .env
TRAEFIK_MODE=upstream
TRAEFIK_ENABLE=true
TRAEFIK_NETWORK=traefik-public
TRAEFIK_DOCKER_NETWORK=traefik-public
MOSAIC_API_DOMAIN=mosaic-api.uscllc.com
MOSAIC_WEB_DOMAIN=mosaic.uscllc.com
MOSAIC_AUTH_DOMAIN=mosaic-auth.uscllc.com
# 4. Create docker-compose.override.yml
cp docker-compose.override.yml.example docker-compose.override.yml
# 5. Edit docker-compose.override.yml - uncomment network section
# networks:
# traefik-public:
# external: true
# name: traefik-public
# 6. Add services to external network
# api:
# networks:
# - traefik-public
# web:
# networks:
# - traefik-public
# authentik-server:
# networks:
# - traefik-public
# 7. Start services
docker compose up -d
```
Services will be auto-discovered by the external Traefik instance.
## Advanced Configuration
### Custom Middleware
Add authentication or rate limiting via Traefik middleware:
```yaml
# docker-compose.override.yml
services:
traefik:
labels:
# Define basic auth middleware
- "traefik.http.middlewares.auth.basicauth.users=admin:$$apr1$$xyz..."
# Define rate limit middleware
- "traefik.http.middlewares.ratelimit.ratelimit.average=100"
- "traefik.http.middlewares.ratelimit.ratelimit.burst=50"
api:
labels:
# Apply middleware to API router
- "traefik.http.routers.mosaic-api.middlewares=auth@docker,ratelimit@docker"
```
Generate basic auth password:
```bash
echo $(htpasswd -nb admin your-password) | sed -e s/\\$/\\$\\$/g
```
### Custom TLS Certificates
To use custom certificates instead of Let's Encrypt:
```yaml
# docker/traefik/dynamic/tls.yml
tls:
certificates:
- certFile: /certs/domain.crt
keyFile: /certs/domain.key
```
Mount certificate directory:
```yaml
# docker-compose.override.yml
services:
traefik:
volumes:
- ./certs:/certs:ro
```
### Multiple Domains
Route multiple domains to different services:
```yaml
# .env
MOSAIC_WEB_DOMAIN=mosaic.local,app.mosaic.local,www.mosaic.local
# Traefik will match all domains
```
Or use override for complex routing:
```yaml
# docker-compose.override.yml
services:
web:
labels:
- "traefik.http.routers.mosaic-web.rule=Host(`mosaic.local`) || Host(`app.mosaic.local`)"
- "traefik.http.routers.mosaic-web-www.rule=Host(`www.mosaic.local`)"
- "traefik.http.routers.mosaic-web-www.middlewares=redirect-www"
- "traefik.http.middlewares.redirect-www.redirectregex.regex=^https://www.mosaic.local/(.*)"
- "traefik.http.middlewares.redirect-www.redirectregex.replacement=https://mosaic.local/$${1}"
```
## Troubleshooting
### Services Not Accessible via Domain
**Check Traefik is running:**
```bash
docker ps | grep traefik
```
**Check Traefik dashboard:**
```bash
# Bundled mode
open http://localhost:8080
# Check registered routers
curl http://localhost:8080/api/http/routers | jq
```
**Verify labels are applied:**
```bash
docker inspect mosaic-api | jq '.Config.Labels'
```
**Check DNS/hosts file:**
```bash
# Local development
cat /etc/hosts | grep mosaic
```
### Certificate Errors
**Self-signed certificates (development):**
- Browser warnings are expected
- Add exception in browser or import CA certificate
**Let's Encrypt failures:**
```bash
# Check Traefik logs
docker logs mosaic-traefik
# Verify ACME email is set
docker exec mosaic-traefik cat /etc/traefik/traefik.yml
# Check certificate storage
docker exec mosaic-traefik ls -la /letsencrypt/
```
### Upstream Mode Not Connecting
**Verify external network exists:**
```bash
docker network ls | grep traefik-public
```
**Create network if missing:**
```bash
docker network create traefik-public
```
**Check service network attachment:**
```bash
docker inspect mosaic-api | jq '.NetworkSettings.Networks'
```
**Verify external Traefik can see services:**
```bash
# From external Traefik container
docker exec <external-traefik-container> traefik healthcheck
```
### Port Conflicts
**Bundled mode port conflicts:**
```bash
# Check what's using ports
sudo lsof -i :80
sudo lsof -i :443
sudo lsof -i :8080
# Change ports in .env
TRAEFIK_HTTP_PORT=8000
TRAEFIK_HTTPS_PORT=8443
TRAEFIK_DASHBOARD_PORT=8081
```
### Dashboard Not Accessible
**Check dashboard is enabled:**
```bash
# In .env
TRAEFIK_DASHBOARD_ENABLED=true
```
**Verify Traefik configuration:**
```bash
docker exec mosaic-traefik cat /etc/traefik/traefik.yml | grep -A5 "api:"
```
**Access dashboard:**
```bash
# Default
http://localhost:8080/dashboard/
# Custom port
http://localhost:${TRAEFIK_DASHBOARD_PORT}/dashboard/
```
## Security Considerations
### Production Checklist
- [ ] Use Let's Encrypt or valid SSL certificates
- [ ] Disable Traefik dashboard or protect with authentication
- [ ] Enable HTTP to HTTPS redirect
- [ ] Configure rate limiting middleware
- [ ] Use strong passwords for all services
- [ ] Restrict Traefik dashboard to internal network
- [ ] Enable Traefik access logs for audit trail
- [ ] Regularly update Traefik image version
### Securing the Dashboard
**Option 1: Disable in production**
```bash
TRAEFIK_DASHBOARD_ENABLED=false
```
**Option 2: Add basic authentication**
```yaml
# docker-compose.override.yml
services:
traefik:
command:
- "--configFile=/etc/traefik/traefik.yml"
- "--api.dashboard=true"
labels:
- "traefik.http.routers.dashboard.rule=Host(`traefik.example.com`)"
- "traefik.http.routers.dashboard.service=api@internal"
- "traefik.http.routers.dashboard.middlewares=auth"
- "traefik.http.middlewares.auth.basicauth.users=admin:$$apr1$$xyz..."
```
**Option 3: IP whitelist**
```yaml
# docker-compose.override.yml
services:
traefik:
labels:
- "traefik.http.middlewares.ipwhitelist.ipwhitelist.sourcerange=127.0.0.1/32,10.0.0.0/8"
- "traefik.http.routers.dashboard.middlewares=ipwhitelist"
```
## Performance Tuning
### Connection Limits
```yaml
# docker/traefik/traefik.yml
entryPoints:
web:
address: ":80"
transport:
respondingTimeouts:
readTimeout: 60
writeTimeout: 60
idleTimeout: 180
websecure:
address: ":443"
transport:
respondingTimeouts:
readTimeout: 60
writeTimeout: 60
idleTimeout: 180
```
### Rate Limiting
```yaml
# docker-compose.override.yml
services:
traefik:
labels:
- "traefik.http.middlewares.ratelimit.ratelimit.average=100"
- "traefik.http.middlewares.ratelimit.ratelimit.burst=200"
- "traefik.http.middlewares.ratelimit.ratelimit.period=1s"
```
## Testing
Integration tests are available to verify Traefik configuration:
```bash
# Run all Traefik tests
./tests/integration/docker/traefik.test.sh all
# Test specific mode
./tests/integration/docker/traefik.test.sh bundled
./tests/integration/docker/traefik.test.sh upstream
./tests/integration/docker/traefik.test.sh none
```
See `tests/integration/docker/README.md` for details.
## Migration Guide
### From None Mode to Bundled Mode
```bash
# 1. Stop existing services
docker compose down
# 2. Backup current .env
cp .env .env.backup
# 3. Switch to bundled configuration
cp .env.traefik-bundled.example .env
# 4. Transfer existing secrets from .env.backup to .env
# 5. Start with Traefik profile
docker compose --profile traefik-bundled up -d
# 6. Update application URLs if needed
# Old: http://localhost:3000
# New: https://mosaic.local
```
### From Bundled Mode to Upstream Mode
```bash
# 1. Ensure external Traefik is running
docker network create traefik-public
# 2. Update .env
TRAEFIK_MODE=upstream
TRAEFIK_NETWORK=traefik-public
# 3. Create override file
cp docker-compose.override.yml.example docker-compose.override.yml
# 4. Edit override file to uncomment network section
# 5. Restart without bundled profile
docker compose down
docker compose up -d
```
## Additional Resources
- [Traefik Official Documentation](https://doc.traefik.io/traefik/)
- [Traefik Docker Provider](https://doc.traefik.io/traefik/providers/docker/)
- [Let's Encrypt with Traefik](https://doc.traefik.io/traefik/https/acme/)
- [Traefik Middleware Reference](https://doc.traefik.io/traefik/middlewares/overview/)