mosaic/stack
1
0
Fork 0
Code Issues 10 Pull Requests Actions Packages Projects Releases 2 Wiki Activity

feat(#93): implement agent spawn via federation

Browse Source 
Implements FED-010: Agent Spawn via Federation feature that enables
spawning and managing Claude agents on remote federated Mosaic Stack
instances via COMMAND message type.

Features:
- Federation agent command types (spawn, status, kill)
- FederationAgentService for handling agent operations
- Integration with orchestrator's agent spawner/lifecycle services
- API endpoints for spawning, querying status, and killing agents
- Full command routing through federation COMMAND infrastructure
- Comprehensive test coverage (12/12 tests passing)

Architecture:
- Hub → Spoke: Spawn agents on remote instances
- Command flow: FederationController → FederationAgentService →
  CommandService → Remote Orchestrator
- Response handling: Remote orchestrator returns agent status/results
- Security: Connection validation, signature verification

Files created:
- apps/api/src/federation/types/federation-agent.types.ts
- apps/api/src/federation/federation-agent.service.ts
- apps/api/src/federation/federation-agent.service.spec.ts

Files modified:
- apps/api/src/federation/command.service.ts (agent command routing)
- apps/api/src/federation/federation.controller.ts (agent endpoints)
- apps/api/src/federation/federation.module.ts (service registration)
- apps/orchestrator/src/api/agents/agents.controller.ts (status endpoint)
- apps/orchestrator/src/api/agents/agents.module.ts (lifecycle integration)

Testing:
- 12/12 tests passing for FederationAgentService
- All command service tests passing
- TypeScript compilation successful
- Linting passed

Refs #93

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Jason Woltje
2026-02-03 14:37:06 -06:00
parent a8c8af21e5
commit 12abdfe81d
405 changed files with 13545 additions and 2153 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/1-getting-started/1-quick-start/1-overview.md
View File

@@ -42,11 +42,11 @@ docker compose logs -f api
## What's Running?
| Service | Port | Purpose |
|---------|------|---------|
| API | 3001 | NestJS backend |
| PostgreSQL | 5432 | Database |
| Valkey | 6379 | Cache (Redis-compatible) |
| Service | Port | Purpose |
| ---------- | ---- | ------------------------ |
| API | 3001 | NestJS backend |
| PostgreSQL | 5432 | Database |
| Valkey | 6379 | Cache (Redis-compatible) |
## Next Steps
@@ -57,6 +57,7 @@ docker compose logs -f api
## Troubleshooting
**Port already in use:**
```bash
# Stop existing services
docker compose down
@@ -66,6 +67,7 @@ lsof -i :3001
```
**Database connection failed:**
```bash
# Check PostgreSQL is running
docker compose ps postgres

1
docs/1-getting-started/2-installation/1-prerequisites.md
View File

@@ -168,5 +168,6 @@ psql --version # 17.x.x or higher (if using native PostgreSQL)
## Next Steps
Proceed to:
- [Local Setup](2-local-setup.md) for native development
- [Docker Setup](3-docker-setup.md) for containerized deployment

3
docs/1-getting-started/2-installation/2-local-setup.md
View File

@@ -20,6 +20,7 @@ pnpm install
```
This installs dependencies for:
- Root workspace
- `apps/api` (NestJS backend)
- `apps/web` (Next.js frontend - when implemented)
@@ -123,6 +124,7 @@ curl http://localhost:3001/health
```
**Expected response:**
```json
{
"status": "ok",
@@ -138,6 +140,7 @@ pnpm test
```
**Expected output:**
```
Test Files 5 passed (5)
Tests 26 passed (26)

26
docs/1-getting-started/2-installation/3-docker-setup.md
View File

@@ -81,17 +81,17 @@ docker compose up -d
**Services available:**
| Service | Container | Port | Profile | Purpose |
|---------|-----------|------|---------|---------|
| PostgreSQL | mosaic-postgres | 5432 | core | Database with pgvector |
| Valkey | mosaic-valkey | 6379 | core | Redis-compatible cache |
| API | mosaic-api | 3001 | core | NestJS backend |
| Web | mosaic-web | 3000 | core | Next.js frontend |
| Authentik Server | mosaic-authentik-server | 9000, 9443 | authentik | OIDC provider |
| Authentik Worker | mosaic-authentik-worker | - | authentik | Background jobs |
| Authentik PostgreSQL | mosaic-authentik-postgres | - | authentik | Auth database |
| Authentik Redis | mosaic-authentik-redis | - | authentik | Auth cache |
| Ollama | mosaic-ollama | 11434 | ollama | LLM service |
| Service | Container | Port | Profile | Purpose |
| -------------------- | ------------------------- | ---------- | --------- | ---------------------- |
| PostgreSQL | mosaic-postgres | 5432 | core | Database with pgvector |
| Valkey | mosaic-valkey | 6379 | core | Redis-compatible cache |
| API | mosaic-api | 3001 | core | NestJS backend |
| Web | mosaic-web | 3000 | core | Next.js frontend |
| Authentik Server | mosaic-authentik-server | 9000, 9443 | authentik | OIDC provider |
| Authentik Worker | mosaic-authentik-worker | - | authentik | Background jobs |
| Authentik PostgreSQL | mosaic-authentik-postgres | - | authentik | Auth database |
| Authentik Redis | mosaic-authentik-redis | - | authentik | Auth cache |
| Ollama | mosaic-ollama | 11434 | ollama | LLM service |
## Step 4: Run Database Migrations
@@ -236,7 +236,7 @@ services:
replicas: 2
resources:
limits:
cpus: '1.0'
cpus: "1.0"
memory: 1G
web:
@@ -247,7 +247,7 @@ services:
replicas: 2
resources:
limits:
cpus: '0.5'
cpus: "0.5"
memory: 512M
```

2
docs/1-getting-started/3-configuration/1-environment.md
View File

@@ -261,11 +261,13 @@ PRISMA_LOG_QUERIES=false
Environment variables are validated at application startup. Missing required variables will cause the application to fail with a clear error message.
**Required variables:**
- `DATABASE_URL`
- `JWT_SECRET`
- `NEXT_PUBLIC_APP_URL`
**Optional variables:**
- All OIDC settings (if using Authentik)
- All Ollama settings (if using AI features)
- Logging and monitoring settings

41
docs/1-getting-started/3-configuration/2-authentik.md
View File

@@ -36,6 +36,7 @@ docker compose logs -f
```
**Access Authentik:**
- URL: http://localhost:9000/if/flow/initial-setup/
- Create admin account during initial setup
@@ -53,17 +54,17 @@ Sign up at [goauthentik.io](https://goauthentik.io) for managed Authentik.
4. **Configure Provider:**
| Field | Value |
|-------|-------|
| **Name** | Mosaic Stack |
| **Authorization flow** | default-provider-authorization-implicit-consent |
| **Client type** | Confidential |
| **Client ID** | (auto-generated, save this) |
| **Client Secret** | (auto-generated, save this) |
| **Redirect URIs** | `http://localhost:3001/auth/callback` |
| **Scopes** | `openid`, `email`, `profile` |
| **Subject mode** | Based on User's UUID |
| **Include claims in id_token** | ✅ Enabled |
| Field | Value |
| ------------------------------ | ----------------------------------------------- |
| **Name** | Mosaic Stack |
| **Authorization flow** | default-provider-authorization-implicit-consent |
| **Client type** | Confidential |
| **Client ID** | (auto-generated, save this) |
| **Client Secret** | (auto-generated, save this) |
| **Redirect URIs** | `http://localhost:3001/auth/callback` |
| **Scopes** | `openid`, `email`, `profile` |
| **Subject mode** | Based on User's UUID |
| **Include claims in id_token** | ✅ Enabled |
5. **Click "Create"**
@@ -77,12 +78,12 @@ Sign up at [goauthentik.io](https://goauthentik.io) for managed Authentik.
3. **Configure Application:**
| Field | Value |
|-------|-------|
| **Name** | Mosaic Stack |
| **Slug** | mosaic-stack |
| **Provider** | Select "Mosaic Stack" (created in Step 2) |
| **Launch URL** | `http://localhost:3000` |
| Field | Value |
| -------------- | ----------------------------------------- |
| **Name** | Mosaic Stack |
| **Slug** | mosaic-stack |
| **Provider** | Select "Mosaic Stack" (created in Step 2) |
| **Launch URL** | `http://localhost:3000` |
4. **Click "Create"**
@@ -99,6 +100,7 @@ OIDC_REDIRECT_URI=http://localhost:3001/auth/callback
```
**Important Notes:**
- `OIDC_ISSUER` must end with a trailing slash `/`
- Replace `<your-client-id>` and `<your-client-secret>` with actual values from Step 2
- `OIDC_REDIRECT_URI` must exactly match what you configured in Authentik
@@ -218,6 +220,7 @@ Customize Authentik's login page:
**Cause:** Redirect URI in `.env` doesn't match Authentik configuration
**Fix:**
```bash
# Ensure exact match (including http vs https)
# In Authentik: http://localhost:3001/auth/callback
@@ -229,6 +232,7 @@ Customize Authentik's login page:
**Cause:** Incorrect client ID or secret
**Fix:**
1. Double-check Client ID and Secret in Authentik provider
2. Copy values exactly (no extra spaces)
3. Update `.env` with correct values
@@ -239,6 +243,7 @@ Customize Authentik's login page:
**Cause:** `OIDC_ISSUER` incorrect or Authentik not accessible
**Fix:**
```bash
# Ensure OIDC_ISSUER ends with /
# Test discovery endpoint
@@ -252,6 +257,7 @@ curl http://localhost:9000/application/o/mosaic-stack/.well-known/openid-configu
**Cause:** User doesn't have permission in Authentik
**Fix:**
1. In Authentik, go to **Directory** → **Users**
2. Select user
3. Click **Assigned to applications**
@@ -264,6 +270,7 @@ Or enable **Superuser privileges** for the user (development only).
**Cause:** JWT expiration set too low
**Fix:**
```bash
# In .env, increase expiration
JWT_EXPIRATION=7d # 7 days instead of 24h

25
docs/1-getting-started/3-configuration/3-docker.md
View File

@@ -93,6 +93,7 @@ OIDC_REDIRECT_URI=http://localhost:3001/auth/callback
```
**Bootstrap Credentials:**
- Username: `akadmin`
- Password: Value of `AUTHENTIK_BOOTSTRAP_PASSWORD`
@@ -124,6 +125,7 @@ COMPOSE_PROFILES=full # Enable all optional services
```
Available profiles:
- `authentik` - Authentik OIDC provider stack
- `ollama` - Ollama LLM service
- `full` - All optional services
@@ -257,7 +259,7 @@ services:
replicas: 2
resources:
limits:
cpus: '1.0'
cpus: "1.0"
memory: 1G
web:
@@ -268,11 +270,12 @@ services:
replicas: 2
resources:
limits:
cpus: '0.5'
cpus: "0.5"
memory: 512M
```
Deploy:
```bash
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
```
@@ -311,9 +314,9 @@ services:
deploy:
resources:
limits:
cpus: '1.0'
cpus: "1.0"
reservations:
cpus: '0.25'
cpus: "0.25"
```
## Health Checks
@@ -325,10 +328,10 @@ All services include health checks. Adjust timing if needed:
services:
postgres:
healthcheck:
interval: 30s # Check every 30s
timeout: 10s # Timeout after 10s
retries: 5 # Retry 5 times
start_period: 60s # Wait 60s before first check
interval: 30s # Check every 30s
timeout: 10s # Timeout after 10s
retries: 5 # Retry 5 times
start_period: 60s # Wait 60s before first check
```
## Logging Configuration
@@ -349,6 +352,7 @@ services:
### Centralized Logging
For production, consider:
- Loki + Grafana
- ELK Stack (Elasticsearch, Logstash, Kibana)
- Fluentd
@@ -371,11 +375,13 @@ services:
### Container Won't Start
Check logs:
```bash
docker compose logs <service>
```
Common issues:
- Port conflict: Change port in `.env`
- Missing environment variable: Check `.env` file
- Health check failing: Increase `start_period`
@@ -383,6 +389,7 @@ Common issues:
### Network Issues
Test connectivity between containers:
```bash
# From API container to PostgreSQL
docker compose exec api sh
@@ -392,6 +399,7 @@ nc -zv postgres 5432
### Volume Permission Issues
Fix permissions:
```bash
# PostgreSQL volume
docker compose exec postgres chown -R postgres:postgres /var/lib/postgresql/data
@@ -400,6 +408,7 @@ docker compose exec postgres chown -R postgres:postgres /var/lib/postgresql/data
### Out of Disk Space
Clean up:
```bash
# Remove unused containers, networks, images
docker system prune -a

20
docs/1-getting-started/4-docker-deployment/README.md
View File

@@ -36,6 +36,7 @@ docker compose logs -f
```
That's it! Your Mosaic Stack is now running:
- Web: http://localhost:3000
- API: http://localhost:3001
- PostgreSQL: localhost:5432
@@ -46,6 +47,7 @@ That's it! Your Mosaic Stack is now running:
Mosaic Stack uses Docker Compose profiles to enable optional services:
### Core Services (Always Active)
- `postgres` - PostgreSQL database
- `valkey` - Valkey cache
- `api` - Mosaic API
@@ -54,6 +56,7 @@ Mosaic Stack uses Docker Compose profiles to enable optional services:
### Optional Services (Profiles)
#### Traefik (Reverse Proxy)
```bash
# Start with bundled Traefik
docker compose --profile traefik-bundled up -d
@@ -63,11 +66,13 @@ COMPOSE_PROFILES=traefik-bundled
```
Services included:
- `traefik` - Traefik reverse proxy with dashboard (http://localhost:8080)
See [Traefik Integration Guide](traefik.md) for detailed configuration options including upstream mode.
#### Authentik (OIDC Provider)
```bash
# Start with Authentik
docker compose --profile authentik up -d
@@ -77,12 +82,14 @@ COMPOSE_PROFILES=authentik
```
Services included:
- `authentik-postgres` - Authentik database
- `authentik-redis` - Authentik cache
- `authentik-server` - Authentik OIDC server (http://localhost:9000)
- `authentik-worker` - Authentik background worker
#### Ollama (AI Service)
```bash
# Start with Ollama
docker compose --profile ollama up -d
@@ -92,9 +99,11 @@ COMPOSE_PROFILES=ollama
```
Services included:
- `ollama` - Ollama LLM service (http://localhost:11434)
#### All Services
```bash
# Start everything
docker compose --profile full up -d
@@ -122,6 +131,7 @@ docker compose --profile full up -d
Use external services for production:
1. Copy override template:
```bash
cp docker-compose.override.yml.example docker-compose.override.yml
```
@@ -145,6 +155,7 @@ Docker automatically merges `docker-compose.yml` and `docker-compose.override.ym
Mosaic Stack uses two Docker networks to organize service communication:
#### mosaic-internal (Backend Services)
- **Purpose**: Isolates database and cache services
- **Services**:
- PostgreSQL (main database)
@@ -159,6 +170,7 @@ Mosaic Stack uses two Docker networks to organize service communication:
- No direct external access to database/cache ports (unless explicitly exposed)
#### mosaic-public (Frontend Services)
- **Purpose**: Services that need external network access
- **Services**:
- Mosaic API (needs to reach Authentik OIDC and external Ollama)
@@ -298,11 +310,13 @@ JWT_SECRET=change-this-to-a-random-secret
### Service Won't Start
Check logs:
```bash
docker compose logs <service-name>
```
Common issues:
- Port already in use: Change port in `.env`
- Health check failing: Wait longer or check service logs
- Missing environment variables: Check `.env` file
@@ -310,11 +324,13 @@ Common issues:
### Database Connection Issues
1. Verify PostgreSQL is healthy:
```bash
docker compose ps postgres
```
2. Check database logs:
```bash
docker compose logs postgres
```
@@ -327,6 +343,7 @@ Common issues:
### Performance Issues
1. Adjust PostgreSQL settings in `.env`:
```bash
POSTGRES_SHARED_BUFFERS=512MB
POSTGRES_EFFECTIVE_CACHE_SIZE=2GB
@@ -334,6 +351,7 @@ Common issues:
```
2. Adjust Valkey memory:
```bash
VALKEY_MAXMEMORY=512mb
```
@@ -394,6 +412,7 @@ docker run --rm -v mosaic-postgres-data:/data -v $(pwd):/backup alpine tar czf /
### Monitoring
Consider adding:
- Prometheus for metrics
- Grafana for dashboards
- Loki for log aggregation
@@ -402,6 +421,7 @@ Consider adding:
### Scaling
For production scaling:
- Use external PostgreSQL (managed service)
- Use external Redis/Valkey cluster
- Load balance multiple API instances

61
docs/1-getting-started/4-docker-deployment/traefik.md
View File

@@ -68,30 +68,30 @@ docker compose up -d
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `TRAEFIK_MODE` | `none` | Traefik mode: `bundled`, `upstream`, or `none` |
| `TRAEFIK_ENABLE` | `false` | Enable Traefik labels on services |
| `MOSAIC_API_DOMAIN` | `api.mosaic.local` | Domain for API service |
| `MOSAIC_WEB_DOMAIN` | `mosaic.local` | Domain for Web service |
| `MOSAIC_AUTH_DOMAIN` | `auth.mosaic.local` | Domain for Authentik service |
| `TRAEFIK_NETWORK` | `traefik-public` | External Traefik network (upstream mode) |
| `TRAEFIK_TLS_ENABLED` | `true` | Enable TLS/HTTPS |
| `TRAEFIK_ACME_EMAIL` | - | Email for Let's Encrypt (production) |
| `TRAEFIK_CERTRESOLVER` | - | Cert resolver name (e.g., `letsencrypt`) |
| `TRAEFIK_DASHBOARD_ENABLED` | `true` | Enable Traefik dashboard (bundled mode) |
| `TRAEFIK_DASHBOARD_PORT` | `8080` | Dashboard port (bundled mode) |
| `TRAEFIK_ENTRYPOINT` | `websecure` | Traefik entrypoint (`web` or `websecure`) |
| `TRAEFIK_DOCKER_NETWORK` | `mosaic-public` | Docker network for Traefik routing |
| Variable | Default | Description |
| --------------------------- | ------------------- | ---------------------------------------------- |
| `TRAEFIK_MODE` | `none` | Traefik mode: `bundled`, `upstream`, or `none` |
| `TRAEFIK_ENABLE` | `false` | Enable Traefik labels on services |
| `MOSAIC_API_DOMAIN` | `api.mosaic.local` | Domain for API service |
| `MOSAIC_WEB_DOMAIN` | `mosaic.local` | Domain for Web service |
| `MOSAIC_AUTH_DOMAIN` | `auth.mosaic.local` | Domain for Authentik service |
| `TRAEFIK_NETWORK` | `traefik-public` | External Traefik network (upstream mode) |
| `TRAEFIK_TLS_ENABLED` | `true` | Enable TLS/HTTPS |
| `TRAEFIK_ACME_EMAIL` | - | Email for Let's Encrypt (production) |
| `TRAEFIK_CERTRESOLVER` | - | Cert resolver name (e.g., `letsencrypt`) |
| `TRAEFIK_DASHBOARD_ENABLED` | `true` | Enable Traefik dashboard (bundled mode) |
| `TRAEFIK_DASHBOARD_PORT` | `8080` | Dashboard port (bundled mode) |
| `TRAEFIK_ENTRYPOINT` | `websecure` | Traefik entrypoint (`web` or `websecure`) |
| `TRAEFIK_DOCKER_NETWORK` | `mosaic-public` | Docker network for Traefik routing |
### Docker Compose Profiles
| Profile | Description |
|---------|-------------|
| Profile | Description |
| ----------------- | --------------------------------- |
| `traefik-bundled` | Activates bundled Traefik service |
| `authentik` | Enables Authentik SSO services |
| `ollama` | Enables Ollama AI service |
| `full` | Enables all optional services |
| `authentik` | Enables Authentik SSO services |
| `ollama` | Enables Ollama AI service |
| `full` | Enables all optional services |
## Deployment Scenarios
@@ -131,6 +131,7 @@ MOSAIC_AUTH_DOMAIN=auth.example.com
```
**Prerequisites:**
1. DNS records pointing to your server
2. Ports 80 and 443 accessible from internet
3. Uncomment ACME configuration in `docker/traefik/traefik.yml`
@@ -216,6 +217,7 @@ services:
```
Generate basic auth password:
```bash
echo $(htpasswd -nb admin your-password) | sed -e s/\\$/\\$\\$/g
```
@@ -233,6 +235,7 @@ tls:
```
Mount certificate directory:
```yaml
# docker-compose.override.yml
services:
@@ -248,7 +251,6 @@ Route multiple domains to different services:
```yaml
# .env
MOSAIC_WEB_DOMAIN=mosaic.local,app.mosaic.local,www.mosaic.local
# Traefik will match all domains
```
@@ -271,11 +273,13 @@ services:
### Services Not Accessible via Domain
**Check Traefik is running:**
```bash
docker ps | grep traefik
```
**Check Traefik dashboard:**
```bash
# Bundled mode
open http://localhost:8080
@@ -285,11 +289,13 @@ curl http://localhost:8080/api/http/routers | jq
```
**Verify labels are applied:**
```bash
docker inspect mosaic-api | jq '.Config.Labels'
```
**Check DNS/hosts file:**
```bash
# Local development
cat /etc/hosts | grep mosaic
@@ -298,10 +304,12 @@ cat /etc/hosts | grep mosaic
### Certificate Errors
**Self-signed certificates (development):**
- Browser warnings are expected
- Add exception in browser or import CA certificate
**Let's Encrypt failures:**
```bash
# Check Traefik logs
docker logs mosaic-traefik
@@ -316,21 +324,25 @@ docker exec mosaic-traefik ls -la /letsencrypt/
### Upstream Mode Not Connecting
**Verify external network exists:**
```bash
docker network ls | grep traefik-public
```
**Create network if missing:**
```bash
docker network create traefik-public
```
**Check service network attachment:**
```bash
docker inspect mosaic-api | jq '.NetworkSettings.Networks'
```
**Verify external Traefik can see services:**
```bash
# From external Traefik container
docker exec <external-traefik-container> traefik healthcheck
@@ -339,6 +351,7 @@ docker exec <external-traefik-container> traefik healthcheck
### Port Conflicts
**Bundled mode port conflicts:**
```bash
# Check what's using ports
sudo lsof -i :80
@@ -354,17 +367,20 @@ TRAEFIK_DASHBOARD_PORT=8081
### Dashboard Not Accessible
**Check dashboard is enabled:**
```bash
# In .env
TRAEFIK_DASHBOARD_ENABLED=true
```
**Verify Traefik configuration:**
```bash
docker exec mosaic-traefik cat /etc/traefik/traefik.yml | grep -A5 "api:"
```
**Access dashboard:**
```bash
# Default
http://localhost:8080/dashboard/
@@ -389,11 +405,13 @@ http://localhost:${TRAEFIK_DASHBOARD_PORT}/dashboard/
### Securing the Dashboard
**Option 1: Disable in production**
```bash
TRAEFIK_DASHBOARD_ENABLED=false
```
**Option 2: Add basic authentication**
```yaml
# docker-compose.override.yml
services:
@@ -409,6 +427,7 @@ services:
```
**Option 3: IP whitelist**
```yaml
# docker-compose.override.yml
services:

2
docs/1-getting-started/README.md
View File

@@ -11,6 +11,7 @@ Complete guide to getting Mosaic Stack installed and configured.
## Prerequisites
Before you begin, ensure you have:
- Node.js 20+ and pnpm 9+
- PostgreSQL 17+ (or Docker)
- Basic familiarity with TypeScript and NestJS
@@ -18,6 +19,7 @@ Before you begin, ensure you have:
## Next Steps
After completing this book, proceed to:
- **Development** — Learn the development workflow
- **Architecture** — Understand the system design
- **API** — Explore the API documentation