diff --git a/docs/design/MS22-DB-CENTRIC-ARCHITECTURE.md b/docs/design/MS22-DB-CENTRIC-ARCHITECTURE.md new file mode 100644 index 0000000..22bc0d0 --- /dev/null +++ b/docs/design/MS22-DB-CENTRIC-ARCHITECTURE.md @@ -0,0 +1,252 @@ +# MS22 Phase 1: DB-Centric Agent Fleet Architecture + +## Design Principles + +1. **Minimal env vars** — Only `DATABASE_URL` and `MOSAIC_SECRET_KEY` needed to start +2. **DB-centric config** — All runtime config lives in Postgres, managed via WebUI +3. **Mosaic is the gatekeeper** — Users never talk to OpenClaw directly +4. **Onboarding-first** — Breakglass user + wizard on first boot, no manual config files +5. **Generic product** — No hardcoded agent names, models, providers, or endpoints + +## Bootstrap Flow + +``` +docker stack deploy (2 env vars) + │ + ▼ +┌─────────────────────┐ +│ Postgres migration │ ← creates tables, no seed data +└─────────────────────┘ + │ + ▼ +┌─────────────────────┐ +│ User opens WebUI │ ← detects empty config +└─────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────┐ +│ ONBOARDING WIZARD │ +│ │ +│ Step 1: Create breakglass admin │ +│ (username + password → bcrypt) │ +│ │ +│ Step 2: Configure OIDC (optional) │ +│ (provider URL, client ID, secret) │ +│ │ +│ Step 3: Add LLM provider │ +│ (type, API key, endpoint, test) │ +│ │ +│ Step 4: Configure agents │ +│ (roles, model assignments) │ +│ Auto-generates gateway tokens │ +│ │ +│ Step 5: Deploy summary + health check │ +└─────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────┐ +│ Agents pick up │ ← GET /api/internal/agent-config/:name +│ config from DB │ +└─────────────────────┘ +``` + +## Auth Layers + +| Flow | Method | Details | +| ------------------------------ | -------------------------- | ----------------------------------------------------- | +| User → Mosaic WebUI | Breakglass (local) or OIDC | Breakglass always available as fallback | +| Mosaic API → OpenClaw | Bearer token | Auto-generated per agent, stored encrypted in DB | +| OpenClaw → Mosaic API (config) | Bearer token | Same agent token, validated by Mosaic | +| OpenClaw → LLM providers | API keys | Stored encrypted in DB, delivered via config endpoint | +| Admin → Settings | RBAC | Admin role required for provider/agent/OIDC config | + +## Database Schema (new tables) + +### `SystemConfig` + +Key-value store for global settings (singleton-ish). + +```prisma +model SystemConfig { + id String @id @default(cuid()) + key String @unique // "oidc.issuerUrl", "oidc.clientId", "onboarding.completed", etc. + value String // plaintext or encrypted (prefix: "enc:") + encrypted Boolean @default(false) + updatedAt DateTime @updatedAt +} +``` + +### `LlmProvider` + +LLM provider configurations. + +```prisma +model LlmProvider { + id String @id @default(cuid()) + name String @unique // "zai", "openai", "anthropic", "ollama-local", etc. + displayName String // "Z.ai", "OpenAI", "Local Ollama" + type String // "zai" | "openai" | "anthropic" | "ollama" | "custom" + baseUrl String? // null for built-in providers, URL for custom/ollama + apiKey String? // encrypted + apiType String @default("openai-completions") // openai-completions | anthropic-messages | etc. + models Json @default("[]") // available model list [{id, name, contextWindow, maxTokens}] + isActive Boolean @default(true) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + agents AgentModelAssignment[] +} +``` + +### `AgentConfig` + +Per-agent configuration (replaces old OpenClawAgent). + +```prisma +model AgentConfig { + id String @id @default(cuid()) + name String @unique // "mosaic-main", "mosaic-projects", etc. + displayName String // "Main Orchestrator", "Projects", etc. + role String // "orchestrator" | "developer" | "researcher" | "operations" + gatewayUrl String // internal Docker URL: "http://mosaic-main:18789" + gatewayToken String // encrypted — auto-generated + isActive Boolean @default(true) + personality String? // SOUL.md content for this agent + toolPermissions Json @default("[]") // allowed tool list + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + modelAssignment AgentModelAssignment? +} +``` + +### `AgentModelAssignment` + +Links agents to providers and models. + +```prisma +model AgentModelAssignment { + id String @id @default(cuid()) + agentId String @unique + agent AgentConfig @relation(fields: [agentId], references: [id]) + providerId String + provider LlmProvider @relation(fields: [providerId], references: [id]) + primaryModel String // "glm-5", "claude-sonnet-4-6", "cogito", etc. + fallbacks Json @default("[]") // [{providerId, model}] + updatedAt DateTime @updatedAt +} +``` + +### `BreakglassUser` + +Local admin user (no OIDC dependency). + +```prisma +model BreakglassUser { + id String @id @default(cuid()) + username String @unique + passwordHash String // bcrypt + isActive Boolean @default(true) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt +} +``` + +## Internal Config Endpoint + +`GET /api/internal/agent-config/:agentName` + +- Auth: Bearer token (agent's own gateway token) +- Returns: Complete `openclaw.json` generated from DB tables +- Includes: model config, provider credentials (decrypted), tool permissions + +## Docker Compose (simplified) + +```yaml +services: + mosaic-api: + image: mosaic/api:latest + environment: + DATABASE_URL: ${DATABASE_URL} + MOSAIC_SECRET_KEY: ${MOSAIC_SECRET_KEY} + + mosaic-web: + image: mosaic/web:latest + environment: + NEXT_PUBLIC_API_URL: http://mosaic-api:4000 + + mosaic-main: + image: alpine/openclaw:latest + command: ["/config/entrypoint.sh"] + environment: + DATABASE_URL: ${DATABASE_URL} + MOSAIC_API_URL: http://mosaic-api:4000 + MOSAIC_SECRET_KEY: ${MOSAIC_SECRET_KEY} + AGENT_NAME: mosaic-main + volumes: + - mosaic-main-state:/home/node/.openclaw + + # Additional agents follow same pattern, only AGENT_NAME differs +``` + +### Entrypoint (simplified) + +```sh +#!/bin/sh +# Fetch config from Mosaic API, write openclaw.json, start gateway +CONFIG=$(curl -sf "${MOSAIC_API_URL}/api/internal/agent-config/${AGENT_NAME}" \ + -H "Authorization: Bearer ${MOSAIC_SECRET_KEY}") +echo "$CONFIG" > /tmp/openclaw.json +export OPENCLAW_CONFIG_PATH=/tmp/openclaw.json +exec openclaw gateway run --bind lan --auth token +``` + +## Task Breakdown + +### Phase 1a: DB Schema + Internal Config API + +- Prisma schema: SystemConfig, LlmProvider, AgentConfig, AgentModelAssignment, BreakglassUser +- Migration +- Internal config endpoint: generates openclaw.json from DB +- Encryption/decryption service for API keys and tokens + +### Phase 1b: Onboarding Wizard (API) + +- Detect first-boot (no breakglass user exists) +- POST /api/onboarding/breakglass — create admin +- POST /api/onboarding/oidc — save OIDC config +- POST /api/onboarding/provider — add LLM provider + test connection +- POST /api/onboarding/agents — configure agent fleet +- POST /api/onboarding/complete — mark onboarding done + +### Phase 1c: Onboarding Wizard (WebUI) + +- Multi-step wizard component +- Breakglass user creation form +- OIDC config form (skip option) +- LLM provider form with connection test +- Agent configuration with model picker +- Summary + deploy health check + +### Phase 1d: Settings Pages (WebUI) + +- Settings/Providers — CRUD for LLM providers +- Settings/Agents — model assignments, personalities, status +- Settings/Auth — OIDC config, breakglass password reset +- All behind admin RBAC + +### Phase 1e: Docker Compose + Entrypoint + +- Simplified compose (AGENT_NAME + shared env vars) +- Entrypoint: curl config from API, write, start +- Health check integration + +### Phase 1f: Chat Proxy + +- Mosaic API routes WebUI chat to correct OpenClaw agent +- SSE streaming pass-through +- Agent selector in WebUI + +## Open Questions + +1. Should agents auto-restart when config changes in DB? (webhook/signal vs polling) +2. Should breakglass user be created via CLI as alternative to WebUI wizard? +3. Config cache TTL in agents? (avoid hitting API on every request)