docs(design): MS22 DB-centric agent fleet architecture
Minimal env vars (DATABASE_URL + MOSAIC_SECRET_KEY), all config in Postgres, onboarding wizard, breakglass auth, OIDC via settings UI.
This commit is contained in:
252
docs/design/MS22-DB-CENTRIC-ARCHITECTURE.md
Normal file
252
docs/design/MS22-DB-CENTRIC-ARCHITECTURE.md
Normal file
@@ -0,0 +1,252 @@
|
||||
# MS22 Phase 1: DB-Centric Agent Fleet Architecture
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. **Minimal env vars** — Only `DATABASE_URL` and `MOSAIC_SECRET_KEY` needed to start
|
||||
2. **DB-centric config** — All runtime config lives in Postgres, managed via WebUI
|
||||
3. **Mosaic is the gatekeeper** — Users never talk to OpenClaw directly
|
||||
4. **Onboarding-first** — Breakglass user + wizard on first boot, no manual config files
|
||||
5. **Generic product** — No hardcoded agent names, models, providers, or endpoints
|
||||
|
||||
## Bootstrap Flow
|
||||
|
||||
```
|
||||
docker stack deploy (2 env vars)
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Postgres migration │ ← creates tables, no seed data
|
||||
└─────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ User opens WebUI │ ← detects empty config
|
||||
└─────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ ONBOARDING WIZARD │
|
||||
│ │
|
||||
│ Step 1: Create breakglass admin │
|
||||
│ (username + password → bcrypt) │
|
||||
│ │
|
||||
│ Step 2: Configure OIDC (optional) │
|
||||
│ (provider URL, client ID, secret) │
|
||||
│ │
|
||||
│ Step 3: Add LLM provider │
|
||||
│ (type, API key, endpoint, test) │
|
||||
│ │
|
||||
│ Step 4: Configure agents │
|
||||
│ (roles, model assignments) │
|
||||
│ Auto-generates gateway tokens │
|
||||
│ │
|
||||
│ Step 5: Deploy summary + health check │
|
||||
└─────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Agents pick up │ ← GET /api/internal/agent-config/:name
|
||||
│ config from DB │
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
## Auth Layers
|
||||
|
||||
| Flow | Method | Details |
|
||||
| ------------------------------ | -------------------------- | ----------------------------------------------------- |
|
||||
| User → Mosaic WebUI | Breakglass (local) or OIDC | Breakglass always available as fallback |
|
||||
| Mosaic API → OpenClaw | Bearer token | Auto-generated per agent, stored encrypted in DB |
|
||||
| OpenClaw → Mosaic API (config) | Bearer token | Same agent token, validated by Mosaic |
|
||||
| OpenClaw → LLM providers | API keys | Stored encrypted in DB, delivered via config endpoint |
|
||||
| Admin → Settings | RBAC | Admin role required for provider/agent/OIDC config |
|
||||
|
||||
## Database Schema (new tables)
|
||||
|
||||
### `SystemConfig`
|
||||
|
||||
Key-value store for global settings (singleton-ish).
|
||||
|
||||
```prisma
|
||||
model SystemConfig {
|
||||
id String @id @default(cuid())
|
||||
key String @unique // "oidc.issuerUrl", "oidc.clientId", "onboarding.completed", etc.
|
||||
value String // plaintext or encrypted (prefix: "enc:")
|
||||
encrypted Boolean @default(false)
|
||||
updatedAt DateTime @updatedAt
|
||||
}
|
||||
```
|
||||
|
||||
### `LlmProvider`
|
||||
|
||||
LLM provider configurations.
|
||||
|
||||
```prisma
|
||||
model LlmProvider {
|
||||
id String @id @default(cuid())
|
||||
name String @unique // "zai", "openai", "anthropic", "ollama-local", etc.
|
||||
displayName String // "Z.ai", "OpenAI", "Local Ollama"
|
||||
type String // "zai" | "openai" | "anthropic" | "ollama" | "custom"
|
||||
baseUrl String? // null for built-in providers, URL for custom/ollama
|
||||
apiKey String? // encrypted
|
||||
apiType String @default("openai-completions") // openai-completions | anthropic-messages | etc.
|
||||
models Json @default("[]") // available model list [{id, name, contextWindow, maxTokens}]
|
||||
isActive Boolean @default(true)
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
agents AgentModelAssignment[]
|
||||
}
|
||||
```
|
||||
|
||||
### `AgentConfig`
|
||||
|
||||
Per-agent configuration (replaces old OpenClawAgent).
|
||||
|
||||
```prisma
|
||||
model AgentConfig {
|
||||
id String @id @default(cuid())
|
||||
name String @unique // "mosaic-main", "mosaic-projects", etc.
|
||||
displayName String // "Main Orchestrator", "Projects", etc.
|
||||
role String // "orchestrator" | "developer" | "researcher" | "operations"
|
||||
gatewayUrl String // internal Docker URL: "http://mosaic-main:18789"
|
||||
gatewayToken String // encrypted — auto-generated
|
||||
isActive Boolean @default(true)
|
||||
personality String? // SOUL.md content for this agent
|
||||
toolPermissions Json @default("[]") // allowed tool list
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
modelAssignment AgentModelAssignment?
|
||||
}
|
||||
```
|
||||
|
||||
### `AgentModelAssignment`
|
||||
|
||||
Links agents to providers and models.
|
||||
|
||||
```prisma
|
||||
model AgentModelAssignment {
|
||||
id String @id @default(cuid())
|
||||
agentId String @unique
|
||||
agent AgentConfig @relation(fields: [agentId], references: [id])
|
||||
providerId String
|
||||
provider LlmProvider @relation(fields: [providerId], references: [id])
|
||||
primaryModel String // "glm-5", "claude-sonnet-4-6", "cogito", etc.
|
||||
fallbacks Json @default("[]") // [{providerId, model}]
|
||||
updatedAt DateTime @updatedAt
|
||||
}
|
||||
```
|
||||
|
||||
### `BreakglassUser`
|
||||
|
||||
Local admin user (no OIDC dependency).
|
||||
|
||||
```prisma
|
||||
model BreakglassUser {
|
||||
id String @id @default(cuid())
|
||||
username String @unique
|
||||
passwordHash String // bcrypt
|
||||
isActive Boolean @default(true)
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
}
|
||||
```
|
||||
|
||||
## Internal Config Endpoint
|
||||
|
||||
`GET /api/internal/agent-config/:agentName`
|
||||
|
||||
- Auth: Bearer token (agent's own gateway token)
|
||||
- Returns: Complete `openclaw.json` generated from DB tables
|
||||
- Includes: model config, provider credentials (decrypted), tool permissions
|
||||
|
||||
## Docker Compose (simplified)
|
||||
|
||||
```yaml
|
||||
services:
|
||||
mosaic-api:
|
||||
image: mosaic/api:latest
|
||||
environment:
|
||||
DATABASE_URL: ${DATABASE_URL}
|
||||
MOSAIC_SECRET_KEY: ${MOSAIC_SECRET_KEY}
|
||||
|
||||
mosaic-web:
|
||||
image: mosaic/web:latest
|
||||
environment:
|
||||
NEXT_PUBLIC_API_URL: http://mosaic-api:4000
|
||||
|
||||
mosaic-main:
|
||||
image: alpine/openclaw:latest
|
||||
command: ["/config/entrypoint.sh"]
|
||||
environment:
|
||||
DATABASE_URL: ${DATABASE_URL}
|
||||
MOSAIC_API_URL: http://mosaic-api:4000
|
||||
MOSAIC_SECRET_KEY: ${MOSAIC_SECRET_KEY}
|
||||
AGENT_NAME: mosaic-main
|
||||
volumes:
|
||||
- mosaic-main-state:/home/node/.openclaw
|
||||
|
||||
# Additional agents follow same pattern, only AGENT_NAME differs
|
||||
```
|
||||
|
||||
### Entrypoint (simplified)
|
||||
|
||||
```sh
|
||||
#!/bin/sh
|
||||
# Fetch config from Mosaic API, write openclaw.json, start gateway
|
||||
CONFIG=$(curl -sf "${MOSAIC_API_URL}/api/internal/agent-config/${AGENT_NAME}" \
|
||||
-H "Authorization: Bearer ${MOSAIC_SECRET_KEY}")
|
||||
echo "$CONFIG" > /tmp/openclaw.json
|
||||
export OPENCLAW_CONFIG_PATH=/tmp/openclaw.json
|
||||
exec openclaw gateway run --bind lan --auth token
|
||||
```
|
||||
|
||||
## Task Breakdown
|
||||
|
||||
### Phase 1a: DB Schema + Internal Config API
|
||||
|
||||
- Prisma schema: SystemConfig, LlmProvider, AgentConfig, AgentModelAssignment, BreakglassUser
|
||||
- Migration
|
||||
- Internal config endpoint: generates openclaw.json from DB
|
||||
- Encryption/decryption service for API keys and tokens
|
||||
|
||||
### Phase 1b: Onboarding Wizard (API)
|
||||
|
||||
- Detect first-boot (no breakglass user exists)
|
||||
- POST /api/onboarding/breakglass — create admin
|
||||
- POST /api/onboarding/oidc — save OIDC config
|
||||
- POST /api/onboarding/provider — add LLM provider + test connection
|
||||
- POST /api/onboarding/agents — configure agent fleet
|
||||
- POST /api/onboarding/complete — mark onboarding done
|
||||
|
||||
### Phase 1c: Onboarding Wizard (WebUI)
|
||||
|
||||
- Multi-step wizard component
|
||||
- Breakglass user creation form
|
||||
- OIDC config form (skip option)
|
||||
- LLM provider form with connection test
|
||||
- Agent configuration with model picker
|
||||
- Summary + deploy health check
|
||||
|
||||
### Phase 1d: Settings Pages (WebUI)
|
||||
|
||||
- Settings/Providers — CRUD for LLM providers
|
||||
- Settings/Agents — model assignments, personalities, status
|
||||
- Settings/Auth — OIDC config, breakglass password reset
|
||||
- All behind admin RBAC
|
||||
|
||||
### Phase 1e: Docker Compose + Entrypoint
|
||||
|
||||
- Simplified compose (AGENT_NAME + shared env vars)
|
||||
- Entrypoint: curl config from API, write, start
|
||||
- Health check integration
|
||||
|
||||
### Phase 1f: Chat Proxy
|
||||
|
||||
- Mosaic API routes WebUI chat to correct OpenClaw agent
|
||||
- SSE streaming pass-through
|
||||
- Agent selector in WebUI
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Should agents auto-restart when config changes in DB? (webhook/signal vs polling)
|
||||
2. Should breakglass user be created via CLI as alternative to WebUI wizard?
|
||||
3. Config cache TTL in agents? (avoid hitting API on every request)
|
||||
Reference in New Issue
Block a user