fix: logs page wired to activity_logs, interceptor optional workspaceId, autoRefresh on
Some checks failed
ci/woodpecker/push/ci Pipeline failed
Some checks failed
ci/woodpecker/push/ci Pipeline failed
This commit is contained in:
1
.claude/worktrees/agent-a56bac50
Submodule
1
.claude/worktrees/agent-a56bac50
Submodule
Submodule .claude/worktrees/agent-a56bac50 added at c15456a779
1
.worktrees/feat-ms21-ui-teams-rbac
Submodule
1
.worktrees/feat-ms21-ui-teams-rbac
Submodule
Submodule .worktrees/feat-ms21-ui-teams-rbac added at c640d22394
1
.worktrees/feat-ms22-openclaw-gateway-module
Submodule
1
.worktrees/feat-ms22-openclaw-gateway-module
Submodule
Submodule .worktrees/feat-ms22-openclaw-gateway-module added at b13ff68e22
@@ -160,8 +160,9 @@ export default function LogsPage(): ReactElement {
|
||||
filters.entityType = entityFilter;
|
||||
}
|
||||
|
||||
const data = await fetchActivityLogs(filters);
|
||||
setActivities(data);
|
||||
const response: Awaited<ReturnType<typeof fetchActivityLogs>> =
|
||||
await fetchActivityLogs(filters);
|
||||
setActivities(response);
|
||||
setError(null);
|
||||
} catch (err: unknown) {
|
||||
console.error("[Logs] Failed to fetch activity logs:", err);
|
||||
|
||||
166
docs/research/00-SUMMARY.md
Normal file
166
docs/research/00-SUMMARY.md
Normal file
@@ -0,0 +1,166 @@
|
||||
# Mosaic Stack — Fast-Track Completion Plan
|
||||
|
||||
**Date:** 2026-03-01
|
||||
**Goal:** Make Mosaic Stack usable for daily agent orchestration in hours, not weeks.
|
||||
|
||||
Based on research of 9 community dashboards (openclaw-dashboard, clawd-control, claw-dashboard, ai-maestro, clawview, clawde-dashboard, agent-web-ui, cogni-flow, openclaw-panel), here is the prioritized build plan.
|
||||
|
||||
---
|
||||
|
||||
## What Mosaic Stack Already Has (Strengths)
|
||||
|
||||
- ✅ Better Auth with CSRF + bearer token bypass for API agents
|
||||
- ✅ NestJS API with PostgreSQL (Prisma), full RBAC
|
||||
- ✅ Next.js 15 web app: dashboard widgets, projects, kanban, calendar, tasks, knowledge, files, logs, terminal (xterm.js+WebSocket), usage tracking, settings
|
||||
- ✅ Agent fleet: agents table, orchestrator endpoint, container lifecycle
|
||||
- ✅ Fleet settings: LLM provider config, agent config
|
||||
|
||||
## What's Missing (Gaps)
|
||||
|
||||
- ❌ Chat page is a stub — not connected to any backend
|
||||
- ❌ No memory/file viewer for agent workspace files
|
||||
- ❌ No cron/automation visibility
|
||||
- ❌ No agent creation wizard — must use DB directly
|
||||
- ❌ Fleet overview lacks real-time status and health indicators
|
||||
- ❌ No rate limiting or audit logging
|
||||
- ❌ No agent-to-agent messaging
|
||||
|
||||
---
|
||||
|
||||
## P0 — Do Today (< 2h each, unblocks daily use)
|
||||
|
||||
### 1. Connect Chat to Backend
|
||||
- **Why:** Chat page exists but does nothing. This is the #1 interaction surface for agents. Without it, Mosaic Stack is a dashboard you look at, not a tool you use.
|
||||
- **Effort:** 2h
|
||||
- **Inspired by:** ai-maestro (agent inbox), clawview (embedded chat)
|
||||
- **Approach:** Wire existing chat UI to WebSocket endpoint. Send messages to agent, display responses. Use existing auth context for user identity. Store messages in PostgreSQL.
|
||||
|
||||
### 2. Fleet Overview with Live Status
|
||||
- **Why:** Can't tell which agents are running, idle, or broken. Every dashboard researched puts this front and center.
|
||||
- **Effort:** 2h
|
||||
- **Inspired by:** clawd-control (card grid), openclaw-dashboard (sparklines)
|
||||
- **Approach:** Agent card grid on fleet page. Each card: name, emoji, status dot (green/yellow/red), last activity, session count. Poll agent health endpoint every 10s. Use existing agents table.
|
||||
|
||||
### 3. Agent Memory/File Viewer
|
||||
- **Why:** Debugging agents requires reading MEMORY.md, HEARTBEAT.md, daily logs. Without this, you SSH into the server every time.
|
||||
- **Effort:** 1-2h
|
||||
- **Inspired by:** openclaw-dashboard (memory viewer with markdown rendering)
|
||||
- **Approach:** NestJS endpoint reads files from agent workspace dir. Path traversal protection. Next.js page: file tree sidebar + markdown preview panel. Read-only initially.
|
||||
|
||||
### 4. Rate Limiting + Security Headers
|
||||
- **Why:** Any exposed web app without rate limiting is a brute-force target. 30 minutes of work prevents real attacks.
|
||||
- **Effort:** 30min
|
||||
- **Inspired by:** openclaw-dashboard (5-attempt lockout, HSTS, CSP)
|
||||
- **Approach:** Add `@nestjs/throttler` to auth endpoints (5 req/min for login). Add `helmet` middleware for security headers.
|
||||
|
||||
### 5. Activity Feed / Recent Events
|
||||
- **Why:** "What happened while I was away?" is the first question every morning. Every dashboard has this.
|
||||
- **Effort:** 1h
|
||||
- **Inspired by:** openclaw-dashboard (live feed via SSE), clawd-control (fleet activity)
|
||||
- **Approach:** Query recent log entries from DB. Display as reverse-chronological list on dashboard. Agent name + action + timestamp. Auto-refresh every 30s.
|
||||
|
||||
---
|
||||
|
||||
## P1 — Do This Week (2-8h each, major features)
|
||||
|
||||
### 6. Agent Creation Wizard
|
||||
- **Why:** Creating agents currently requires direct DB manipulation. Friction kills adoption.
|
||||
- **Effort:** 3-4h
|
||||
- **Inspired by:** clawd-control (guided wizard), ai-maestro (UI-based agent creation)
|
||||
- **Approach:** Dialog/wizard in fleet settings: name, emoji, model, connection details (host/port/token), workspace path. Writes to agents table. Could be single-page form (faster) or multi-step (nicer UX).
|
||||
|
||||
### 7. Cron/Automation Management
|
||||
- **Why:** Scheduled tasks are invisible — you don't know what's running, when, or if it failed.
|
||||
- **Effort:** 2-3h
|
||||
- **Inspired by:** openclaw-dashboard (cron list with toggle/trigger)
|
||||
- **Approach:** NestJS reads scheduled jobs (from @nestjs/schedule or config). API: list, toggle, trigger. Frontend: table with Name | Schedule | Status | Last Run | Actions.
|
||||
|
||||
### 8. Audit Logging
|
||||
- **Why:** Security compliance and debugging. "Who did what, when?" is unanswerable without this.
|
||||
- **Effort:** 2-3h
|
||||
- **Inspired by:** openclaw-dashboard (audit.log with auto-rotation)
|
||||
- **Approach:** NestJS middleware logs auth events, destructive actions, config changes to audit_logs table. View in Settings > Security.
|
||||
|
||||
### 9. Agent-to-Agent Simple Messaging
|
||||
- **Why:** Orchestrating multiple agents requires passing context between them. Without messaging, the human is the bottleneck.
|
||||
- **Effort:** 4-6h
|
||||
- **Inspired by:** ai-maestro (AMP protocol — simplified)
|
||||
- **Approach:** `messages` table in PostgreSQL: fromAgentId, toAgentId, type, priority, subject, body, threadId, readAt. API endpoints for send/list/read. Agent inbox UI. Skip cryptographic signing and multi-machine for now.
|
||||
|
||||
### 10. SSE for Real-Time Fleet Updates
|
||||
- **Why:** Polling is fine initially but SSE gives instant feedback when agents change state.
|
||||
- **Effort:** 2-3h
|
||||
- **Inspired by:** openclaw-dashboard, clawd-control (both use SSE)
|
||||
- **Approach:** NestJS SSE endpoint streams agent status changes. Next.js EventSource client updates fleet cards in real-time.
|
||||
|
||||
---
|
||||
|
||||
## P2 — Nice to Have (8h+, polish)
|
||||
|
||||
### 11. TOTP Multi-Factor Authentication
|
||||
- **Effort:** 4-6h
|
||||
- **Inspired by:** openclaw-dashboard
|
||||
- **Approach:** Better Auth may have a TOTP plugin. Otherwise use `otplib` + QR code generation.
|
||||
|
||||
### 12. Multi-Machine Agent Mesh
|
||||
- **Effort:** 16h+
|
||||
- **Inspired by:** ai-maestro (peer mesh, no central server)
|
||||
- **Approach:** Agent discovery across machines. Network-aware routing. Defer until single-machine is solid.
|
||||
|
||||
### 13. Code Graph / Codebase Visualization
|
||||
- **Effort:** 12h+
|
||||
- **Inspired by:** ai-maestro (interactive code graph with delta indexing)
|
||||
- **Approach:** Use ts-morph to parse codebase, D3.js for visualization. Cool but not urgent.
|
||||
|
||||
### 14. Activity Heatmap
|
||||
- **Effort:** 4h
|
||||
- **Inspired by:** openclaw-dashboard (30-day heatmap)
|
||||
- **Approach:** GitHub-style contribution heatmap showing agent activity by hour/day.
|
||||
|
||||
### 15. Agent Personality Profiles
|
||||
- **Effort:** 2-3h
|
||||
- **Inspired by:** ai-maestro (avatars, personality, visual identity)
|
||||
- **Approach:** Add personality/system-prompt field to agent config. Avatar upload. Nice for team feel.
|
||||
|
||||
---
|
||||
|
||||
## Execution Order (Recommended)
|
||||
|
||||
```
|
||||
Day 1 (Today):
|
||||
Morning: #4 Rate limiting (30min) → #2 Fleet overview (2h)
|
||||
Afternoon: #1 Connect chat (2h) → #3 Memory viewer (1.5h)
|
||||
Evening: #5 Activity feed (1h)
|
||||
|
||||
Day 2-3:
|
||||
#6 Agent creation wizard (3h)
|
||||
#7 Cron management (2h)
|
||||
#8 Audit logging (2h)
|
||||
|
||||
Day 4-5:
|
||||
#9 Agent messaging (5h)
|
||||
#10 SSE real-time (2h)
|
||||
|
||||
Week 2+:
|
||||
P2 items as time permits
|
||||
```
|
||||
|
||||
## Total Effort to "Usable Daily"
|
||||
|
||||
| Priority | Items | Total Hours |
|
||||
|----------|-------|-------------|
|
||||
| P0 | 5 items | ~7h |
|
||||
| P1 | 5 items | ~15h |
|
||||
| P2 | 5 items | ~40h+ |
|
||||
|
||||
**Bottom line:** ~7 hours of focused work today gets Mosaic Stack from "demo" to "daily driver." Another 15 hours this week makes it genuinely powerful. The P2 items are polish — nice but not blocking daily use.
|
||||
|
||||
---
|
||||
|
||||
## Key Design Principles (Learned from Research)
|
||||
|
||||
1. **Simplicity first** (clawd-control) — No build tools for simple features. Use what's already there.
|
||||
2. **Single-screen overview** (all dashboards) — Users want one page that answers "is everything OK?"
|
||||
3. **Read before write** (openclaw-dashboard) — Memory viewer is read-only first, edit later.
|
||||
4. **Progressive enhancement** — Polling → SSE → WebSocket. Don't over-engineer day one.
|
||||
5. **Existing infra** — PostgreSQL, NestJS, Next.js are already set up. Don't add new databases or frameworks.
|
||||
721
docs/research/01-chat-orchestration-research.md
Normal file
721
docs/research/01-chat-orchestration-research.md
Normal file
@@ -0,0 +1,721 @@
|
||||
# Chat Interface + Task Orchestration Research Report
|
||||
|
||||
**Date:** 2026-03-01
|
||||
**Focus:** Analysis of Mission Control and Clawtrol for Mosaic Stack feature development
|
||||
**Goal:** Extract actionable design patterns for chat, task dispatch, and live event feeds
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Both Mission Control and Clawtrol are OpenClaw-compatible dashboards with complementary strengths:
|
||||
|
||||
| Feature | Mission Control | Clawtrol | Mosaic Stack Gap |
|
||||
|---------|----------------|----------|------------------|
|
||||
| Chat with agents | ❌ No direct chat | ✅ Full session chat + send | **HIGH** - Stub exists, not wired |
|
||||
| Task dispatch | ✅ AI planning + Kanban | ✅ Simple Kanban | Medium - Kanban exists |
|
||||
| Live events | ✅ SSE-based feed | ❌ Polling only | Medium - SSE polling exists |
|
||||
| Session viewer | ❌ No | ✅ Full transcript view | **HIGH** - Missing |
|
||||
| Agent management | ✅ Auto-create agents | ❌ Basic list | Medium |
|
||||
|
||||
**Top 3 Quick Wins for Mosaic Stack:**
|
||||
1. **Session chat interface** (< 4 hours) - Wire existing chat stub to OpenClaw API
|
||||
2. **Session list view** (< 2 hours) - Read `sessions.json` + `.jsonl` transcripts
|
||||
3. **Task card planning indicator** (< 1 hour) - Add purple pulse animation
|
||||
|
||||
---
|
||||
|
||||
## 1. Chat Interface Analysis
|
||||
|
||||
### Clawtrol Sessions Module (Best Reference)
|
||||
|
||||
**File:** `src/components/modules/SessionsModule/index.tsx`
|
||||
|
||||
**Key Architecture:**
|
||||
```typescript
|
||||
// Session list fetched from OpenClaw
|
||||
const res = await fetch('/api/sessions');
|
||||
const data = await res.json();
|
||||
setSessions(data.sessions || []);
|
||||
|
||||
// Session detail with message history
|
||||
const res = await fetch(`/api/sessions/${encodeURIComponent(session.key)}?limit=50`);
|
||||
const data = await res.json();
|
||||
setChatMessages(data.messages || []);
|
||||
|
||||
// Send message to session (via Telegram or direct)
|
||||
await fetch('/api/sessions/send', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ sessionKey: selectedSession.key, message: msg }),
|
||||
});
|
||||
```
|
||||
|
||||
**UI Pattern - Two-Column Chat Layout:**
|
||||
```tsx
|
||||
// Session list view
|
||||
<div className="grid grid-cols-1 lg:grid-cols-2 xl:grid-cols-3 gap-3">
|
||||
{sessions.map(session => (
|
||||
<div onClick={() => openSessionChat(session)}>
|
||||
{/* Activity indicator */}
|
||||
<div className="w-2 h-2 rounded-full"
|
||||
style={{ background: activityColor, boxShadow: '0 0 6px ...' }} />
|
||||
|
||||
{/* Session metadata */}
|
||||
<span>{session.messageCount} msgs · {session.totalTokens}k tokens</span>
|
||||
<span>${session.estimatedCost.toFixed(2)}</span>
|
||||
|
||||
{/* Last message preview */}
|
||||
<div className="truncate">
|
||||
{session.lastMessages[0]?.text?.slice(0, 100)}
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
```
|
||||
|
||||
**Chat View Pattern:**
|
||||
```tsx
|
||||
// Messages container with auto-scroll
|
||||
<div className="flex-1 overflow-auto p-4 space-y-3">
|
||||
{chatMessages.map(msg => (
|
||||
<div className={msg.role === 'user' ? 'justify-end' : 'justify-start'}>
|
||||
<div className="max-w-[85%] rounded-lg px-3 py-2">
|
||||
{/* Role badge */}
|
||||
<span className="text-[9px] uppercase">
|
||||
{msg.role === 'user' ? 'you' : 'assistant'}
|
||||
</span>
|
||||
|
||||
{/* Markdown content */}
|
||||
<div>{renderMarkdown(msg.text)}</div>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
<div ref={chatEndRef} /> {/* Auto-scroll anchor */}
|
||||
</div>
|
||||
|
||||
// Input with Enter to send
|
||||
<input onKeyDown={e => e.key === 'Enter' && sendChatMessage()} />
|
||||
```
|
||||
|
||||
**Session API Pattern (`/api/sessions/route.ts`):**
|
||||
```typescript
|
||||
// Priority: CLI > Index file > Direct file scan
|
||||
const SESSIONS_INDEX = join(os.homedir(), '.openclaw', 'agents', 'main', 'sessions', 'sessions.json');
|
||||
const SESSIONS_DIR = join(os.homedir(), '.openclaw', 'agents', 'main', 'sessions');
|
||||
|
||||
// Read sessions from index
|
||||
const sessionsMap = JSON.parse(await readFile(SESSIONS_INDEX, 'utf-8'));
|
||||
|
||||
// Enrich with message count and last messages
|
||||
for (const session of sessions) {
|
||||
const [msgs, count] = await Promise.all([
|
||||
getLastMessages(sessionFile, 3), // Last 3 messages
|
||||
getMessageCount(sessionFile), // Total count
|
||||
]);
|
||||
}
|
||||
|
||||
// Parse JSONL for messages
|
||||
function getLastMessages(sessionFile: string, count: number) {
|
||||
const lines = data.trim().split('\n').filter(Boolean);
|
||||
for (let i = lines.length - 1; i >= 0 && messages.length < count; i--) {
|
||||
const parsed = JSON.parse(lines[i]);
|
||||
if (parsed.type === 'message' && parsed.message) {
|
||||
messages.unshift({
|
||||
role: parsed.message.role,
|
||||
text: extractTextFromContent(parsed.message.content),
|
||||
timestamp: parsed.timestamp,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Message Send Pattern (`/api/sessions/send/route.ts`):**
|
||||
```typescript
|
||||
// Parse session key to determine target
|
||||
function parseSessionKey(key: string): { chatId: string; topicId?: string } | null {
|
||||
// agent:main:main → DM to owner
|
||||
if (key === 'agent:main:main') {
|
||||
return { chatId: await getDefaultChatId() };
|
||||
}
|
||||
|
||||
// agent:main:telegram:group:<id>:topic:<id>
|
||||
const topicMatch = key.match(/:group:(-?\d+):topic:(\d+)$/);
|
||||
if (topicMatch) {
|
||||
return { chatId: topicMatch[1], topicId: topicMatch[2] };
|
||||
}
|
||||
}
|
||||
|
||||
// Send via Telegram Bot API (or could use OpenClaw chat.send)
|
||||
const res = await fetch(`https://api.telegram.org/bot${botToken}/sendMessage`, {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ chat_id: target.chatId, text: message }),
|
||||
});
|
||||
```
|
||||
|
||||
### Key Takeaways for Mosaic Stack
|
||||
|
||||
1. **Session key format:** `agent:main:telegram:group:<id>:topic:<id>` or `agent:main:main`
|
||||
2. **JSONL parsing:** Read from `~/.openclaw/agents/main/sessions/<session-id>.jsonl`
|
||||
3. **Cost estimation:**
|
||||
```typescript
|
||||
const isOpus = modelName.includes('opus');
|
||||
const inputRate = isOpus ? 15 : 3;
|
||||
const outputRate = isOpus ? 75 : 15;
|
||||
const cost = (inputTokens / 1_000_000 * inputRate) + (outputTokens / 1_000_000 * outputRate);
|
||||
```
|
||||
4. **Activity color logic:**
|
||||
```typescript
|
||||
if (lastActivity > hourAgo) return 'green'; // Active
|
||||
if (lastActivity > dayAgo) return 'yellow'; // Recent
|
||||
return 'dim'; // Stale
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Task/Agent Dispatch Flow (Mission Control)
|
||||
|
||||
### AI Planning UX Pattern
|
||||
|
||||
**The Flow:**
|
||||
```
|
||||
CREATE → PLAN (AI Q&A) → ASSIGN (Auto-agent) → EXECUTE → DELIVER
|
||||
```
|
||||
|
||||
**Status Columns:**
|
||||
```
|
||||
PLANNING → INBOX → ASSIGNED → IN PROGRESS → TESTING → REVIEW → DONE
|
||||
```
|
||||
|
||||
**PlanningTab.tsx - Core Pattern:**
|
||||
|
||||
1. **Start Planning Button:**
|
||||
```tsx
|
||||
if (!state?.isStarted) {
|
||||
return (
|
||||
<button onClick={startPlanning} className="px-6 py-3 bg-mc-accent">
|
||||
📋 Start Planning
|
||||
</button>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
2. **Question/Answer Loop:**
|
||||
```tsx
|
||||
// Current question display
|
||||
<h3>{state.currentQuestion.question}</h3>
|
||||
|
||||
// Multiple choice options
|
||||
{state.currentQuestion.options.map(option => (
|
||||
<button
|
||||
onClick={() => setSelectedOption(option.label)}
|
||||
className={isSelected ? 'border-mc-accent bg-mc-accent/10' : 'border-mc-border'}
|
||||
>
|
||||
<span className="w-8 h-8">{option.id.toUpperCase()}</span>
|
||||
<span>{option.label}</span>
|
||||
{isSelected && <CheckCircle />}
|
||||
</button>
|
||||
))}
|
||||
|
||||
// "Other" option with text input
|
||||
{isOther && isSelected && (
|
||||
<input placeholder="Please specify..." value={otherText} />
|
||||
)}
|
||||
```
|
||||
|
||||
3. **Polling for AI Response:**
|
||||
```typescript
|
||||
// Poll every 2 seconds for next question
|
||||
pollingIntervalRef.current = setInterval(() => {
|
||||
pollForUpdates();
|
||||
}, 2000);
|
||||
|
||||
// 90-second timeout
|
||||
pollingTimeoutRef.current = setTimeout(() => {
|
||||
setError('Taking too long to respond...');
|
||||
}, 90000);
|
||||
```
|
||||
|
||||
4. **Planning Complete - Spec Display:**
|
||||
```tsx
|
||||
if (state?.isComplete && state?.spec) {
|
||||
return (
|
||||
<div>
|
||||
<div className="flex items-center gap-2 text-green-400">
|
||||
<Lock /> Planning Complete
|
||||
</div>
|
||||
|
||||
{/* Generated spec */}
|
||||
<div className="bg-mc-bg border rounded-lg p-4">
|
||||
<h3>{state.spec.title}</h3>
|
||||
<p>{state.spec.summary}</p>
|
||||
<ul>{state.spec.deliverables.map(d => <li>{d}</li>)}</ul>
|
||||
<ul>{state.spec.success_criteria.map(c => <li>{c}</li>)}</ul>
|
||||
</div>
|
||||
|
||||
{/* Auto-created agents */}
|
||||
{state.agents.map(agent => (
|
||||
<div className="flex items-center gap-3">
|
||||
<span className="text-2xl">{agent.avatar_emoji}</span>
|
||||
<div>
|
||||
<p>{agent.name}</p>
|
||||
<p className="text-sm">{agent.role}</p>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Planning API Pattern
|
||||
|
||||
**POST `/api/tasks/[id]/planning` - Start Planning:**
|
||||
```typescript
|
||||
// Create session key
|
||||
const sessionKey = `agent:main:planning:${taskId}`;
|
||||
|
||||
// Build planning prompt
|
||||
const planningPrompt = `
|
||||
PLANNING REQUEST
|
||||
|
||||
Task Title: ${task.title}
|
||||
Task Description: ${task.description}
|
||||
|
||||
Generate your FIRST question. Respond with ONLY valid JSON:
|
||||
{
|
||||
"question": "Your question here?",
|
||||
"options": [
|
||||
{"id": "A", "label": "First option"},
|
||||
{"id": "B", "label": "Second option"},
|
||||
{"id": "other", "label": "Other"}
|
||||
]
|
||||
}
|
||||
`;
|
||||
|
||||
// Send to OpenClaw
|
||||
await client.call('chat.send', {
|
||||
sessionKey,
|
||||
message: planningPrompt,
|
||||
});
|
||||
|
||||
// Store in DB
|
||||
UPDATE tasks SET planning_session_key = ?, planning_messages = ?, status = 'planning'
|
||||
```
|
||||
|
||||
**Key Insight:** The AI doesn't just plan - it asks **multiple-choice questions** to clarify requirements. This is the "AI clarification before dispatch" pattern.
|
||||
|
||||
### Kanban Card with Planning Indicator
|
||||
|
||||
```tsx
|
||||
// TaskCard.tsx
|
||||
const isPlanning = task.status === 'planning';
|
||||
|
||||
<div className={isPlanning
|
||||
? 'border-purple-500/40 hover:border-purple-500'
|
||||
: 'border-mc-border/50 hover:border-mc-accent/40'}>
|
||||
|
||||
{isPlanning && (
|
||||
<div className="flex items-center gap-2 py-2 px-3 bg-purple-500/10">
|
||||
<div className="w-2 h-2 bg-purple-500 rounded-full animate-pulse" />
|
||||
<span className="text-xs text-purple-400">Continue planning</span>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
```
|
||||
|
||||
### Auto-Dispatch Pattern
|
||||
|
||||
```typescript
|
||||
// When task moves from PLANNING → INBOX (planning complete)
|
||||
if (shouldTriggerAutoDispatch(oldStatus, newStatus, agentId)) {
|
||||
await triggerAutoDispatch({
|
||||
taskId,
|
||||
taskTitle,
|
||||
agentId,
|
||||
agentName,
|
||||
workspaceId,
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Live Event Feed
|
||||
|
||||
### Mission Control SSE Pattern
|
||||
|
||||
**`src/lib/events.ts`:**
|
||||
```typescript
|
||||
// In-memory client registry
|
||||
const clients = new Set<ReadableStreamDefaultController>();
|
||||
|
||||
export function registerClient(controller) {
|
||||
clients.add(controller);
|
||||
}
|
||||
|
||||
export function broadcast(event: SSEEvent) {
|
||||
const data = `data: ${JSON.stringify(event)}\n\n`;
|
||||
const encoded = new TextEncoder().encode(data);
|
||||
|
||||
for (const client of Array.from(clients)) {
|
||||
try {
|
||||
client.enqueue(encoded);
|
||||
} catch {
|
||||
clients.delete(client);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**LiveFeed Component:**
|
||||
```tsx
|
||||
// Filter tabs
|
||||
<div className="flex gap-1">
|
||||
{['all', 'tasks', 'agents'].map(tab => (
|
||||
<button className={filter === tab ? 'bg-mc-accent' : ''}>
|
||||
{tab}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
// Event list with icons
|
||||
{filteredEvents.map(event => (
|
||||
<div className={`p-2 rounded border-l-2 ${
|
||||
isHighlight ? 'bg-mc-bg-tertiary border-mc-accent-pink' : 'hover:bg-mc-bg-tertiary'
|
||||
}`}>
|
||||
<span>{getEventIcon(event.type)}</span>
|
||||
<p>{event.message}</p>
|
||||
<span className="text-xs">{formatDistanceToNow(event.created_at)}</span>
|
||||
</div>
|
||||
))}
|
||||
|
||||
// Event icons
|
||||
function getEventIcon(type: string) {
|
||||
switch (type) {
|
||||
case 'task_created': return '📋';
|
||||
case 'task_assigned': return '👤';
|
||||
case 'task_completed': return '✅';
|
||||
case 'message_sent': return '💬';
|
||||
case 'agent_joined': return '🎉';
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### SSE vs WebSocket Trade-off
|
||||
|
||||
| Aspect | SSE (Mission Control) | WebSocket (Clawtrol) |
|
||||
|--------|----------------------|---------------------|
|
||||
| Direction | Server → Client only | Bidirectional |
|
||||
| Reconnect | Automatic browser handling | Manual implementation |
|
||||
| Overhead | HTTP-based, lighter | Full TCP connection |
|
||||
| Use case | Event feeds, notifications | Real-time terminal, chat |
|
||||
|
||||
**Recommendation:** Use SSE for event feeds (simpler), WebSocket for interactive terminals.
|
||||
|
||||
---
|
||||
|
||||
## 4. Session Viewer Pattern
|
||||
|
||||
### Clawtrol Session List
|
||||
|
||||
```tsx
|
||||
// Session card with activity indicator
|
||||
<div className="card-base cursor-pointer" onClick={() => openSessionChat(session)}>
|
||||
{/* Activity dot */}
|
||||
<div className="w-2 h-2 rounded-full"
|
||||
style={{
|
||||
background: activityColor,
|
||||
boxShadow: activityColor === 'green' ? '0 0 6px rgba(0,255,106,0.5)' : undefined
|
||||
}} />
|
||||
|
||||
{/* Session info */}
|
||||
<h3 className="truncate">{session.label}</h3>
|
||||
<div className="text-[9px]">
|
||||
{session.messageCount} msgs · {session.totalTokens}k tokens
|
||||
{session.estimatedCost > 0 && <span> · ${session.estimatedCost.toFixed(2)}</span>}
|
||||
{session.model && <span> · {session.model}</span>}
|
||||
</div>
|
||||
|
||||
{/* Last message preview */}
|
||||
{session.lastMessages?.length > 0 && (
|
||||
<div className="mt-2 p-2 rounded bg-secondary">
|
||||
<span>{session.lastMessages[0]?.role === 'user' ? 'you: ' : 'assistant: '}</span>
|
||||
<span className="truncate">{session.lastMessages[0]?.text?.slice(0, 100)}</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
```
|
||||
|
||||
### Session Label Mapping
|
||||
|
||||
```typescript
|
||||
const TOPIC_NAMES: Record<string, string> = {
|
||||
'1369': '🔖 Bookmarks',
|
||||
'13': '🌴 Bali Trip',
|
||||
'14': '💰 Expenses',
|
||||
// ... user-defined topic labels
|
||||
};
|
||||
|
||||
function getSessionLabel(key: string): string {
|
||||
if (key === 'agent:main:main') return 'Main Session (DM)';
|
||||
if (key.includes(':subagent:')) return `Subagent ${uuid.slice(0, 8)}`;
|
||||
|
||||
// Telegram topic
|
||||
const topicMatch = key.match(/:topic:(\d+)$/);
|
||||
if (topicMatch) {
|
||||
return TOPIC_NAMES[topicMatch[1]] || `Topic ${topicMatch[1]}`;
|
||||
}
|
||||
|
||||
return key.split(':').pop() || key;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. OpenClaw Client Integration
|
||||
|
||||
### WebSocket Client Pattern
|
||||
|
||||
**`src/lib/openclaw/client.ts`:**
|
||||
```typescript
|
||||
export class OpenClawClient extends EventEmitter {
|
||||
private ws: WebSocket | null = null;
|
||||
private pendingRequests = new Map<string, PromiseHandlers>();
|
||||
private connected = false;
|
||||
private authenticated = false;
|
||||
|
||||
async connect(): Promise<void> {
|
||||
// Add token to URL for auth
|
||||
const wsUrl = new URL(this.url);
|
||||
wsUrl.searchParams.set('token', this.token);
|
||||
|
||||
this.ws = new WebSocket(wsUrl.toString());
|
||||
|
||||
this.ws.onmessage = (event) => {
|
||||
const data = JSON.parse(event.data);
|
||||
|
||||
// Handle challenge-response auth
|
||||
if (data.type === 'event' && data.event === 'connect.challenge') {
|
||||
const response = {
|
||||
type: 'req',
|
||||
id: crypto.randomUUID(),
|
||||
method: 'connect',
|
||||
params: {
|
||||
auth: { token: this.token },
|
||||
role: 'operator',
|
||||
scopes: ['operator.admin'],
|
||||
}
|
||||
};
|
||||
this.ws.send(JSON.stringify(response));
|
||||
return;
|
||||
}
|
||||
|
||||
// Handle RPC responses
|
||||
if (data.type === 'res') {
|
||||
const pending = this.pendingRequests.get(data.id);
|
||||
if (pending) {
|
||||
data.ok ? pending.resolve(data.payload) : pending.reject(data.error);
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
async call<T>(method: string, params?: object): Promise<T> {
|
||||
const id = crypto.randomUUID();
|
||||
const message = { type: 'req', id, method, params };
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
this.pendingRequests.set(id, { resolve, reject });
|
||||
this.ws.send(JSON.stringify(message));
|
||||
|
||||
// 30s timeout
|
||||
setTimeout(() => {
|
||||
if (this.pendingRequests.has(id)) {
|
||||
this.pendingRequests.delete(id);
|
||||
reject(new Error(`Timeout: ${method}`));
|
||||
}
|
||||
}, 30000);
|
||||
});
|
||||
}
|
||||
|
||||
// Convenience methods
|
||||
async listSessions() { return this.call('sessions.list'); }
|
||||
async sendMessage(sessionId: string, content: string) {
|
||||
return this.call('sessions.send', { session_id: sessionId, content });
|
||||
}
|
||||
async listAgents() { return this.call('agents.list'); }
|
||||
}
|
||||
```
|
||||
|
||||
### Event Deduplication Pattern
|
||||
|
||||
```typescript
|
||||
// Global dedup cache (survives Next.js hot reload)
|
||||
const GLOBAL_EVENT_CACHE_KEY = '__openclaw_processed_events__';
|
||||
const globalProcessedEvents = globalThis[GLOBAL_EVENT_CACHE_KEY] || new Map();
|
||||
|
||||
// Content-based event ID
|
||||
function generateEventId(data: any): string {
|
||||
const canonical = JSON.stringify({
|
||||
type: data.type,
|
||||
seq: data.seq,
|
||||
runId: data.payload?.runId,
|
||||
payloadHash: createHash('sha256').update(JSON.stringify(data.payload)).digest('hex').slice(0, 16),
|
||||
});
|
||||
return createHash('sha256').update(canonical).digest('hex').slice(0, 32);
|
||||
}
|
||||
|
||||
// Skip duplicates
|
||||
if (globalProcessedEvents.has(eventId)) return;
|
||||
globalProcessedEvents.set(eventId, Date.now());
|
||||
|
||||
// LRU cleanup
|
||||
if (globalProcessedEvents.size > MAX_EVENTS) {
|
||||
// Remove oldest entries
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Feature Recommendations for Mosaic Stack
|
||||
|
||||
### Quick Wins (< 4 hours each)
|
||||
|
||||
| Feature | Effort | Impact | Source |
|
||||
|---------|--------|--------|--------|
|
||||
| **Session list page** | 2h | HIGH | Clawtrol |
|
||||
| **Session chat interface** | 4h | HIGH | Clawtrol |
|
||||
| **Planning indicator on task cards** | 1h | MEDIUM | Mission Control |
|
||||
| **Activity dots (green/yellow/dim)** | 30m | MEDIUM | Clawtrol |
|
||||
| **Token/cost display per session** | 1h | MEDIUM | Clawtrol |
|
||||
| **Event feed filter tabs** | 1h | LOW | Mission Control |
|
||||
|
||||
### Medium Effort (4-16 hours)
|
||||
|
||||
| Feature | Effort | Impact | Description |
|
||||
|---------|--------|--------|-------------|
|
||||
| **AI planning flow** | 8h | HIGH | Multi-choice Q&A before dispatch |
|
||||
| **OpenClaw WebSocket client** | 4h | HIGH | Real-time event streaming |
|
||||
| **Session transcript viewer** | 4h | MEDIUM | JSONL parsing + display |
|
||||
| **Auto-agent creation** | 8h | MEDIUM | Generate agents from planning spec |
|
||||
|
||||
### Architecture Recommendations
|
||||
|
||||
1. **Keep SSE for event feed** - Simpler than WebSocket for one-way updates
|
||||
2. **Use OpenClaw `chat.send` for messages** - Don't implement Telegram API directly
|
||||
3. **Store session metadata in PostgreSQL** - Mirror `sessions.json` for joins
|
||||
4. **Implement planning as a state machine** - Clear states: idle → started → questioning → complete
|
||||
|
||||
---
|
||||
|
||||
## 7. Code Snippets to Reuse
|
||||
|
||||
### Session API Route (Clawtrol-style)
|
||||
|
||||
```typescript
|
||||
// app/api/sessions/route.ts
|
||||
import { readFile, readdir } from 'fs/promises';
|
||||
import { join } from 'path';
|
||||
import os from 'os';
|
||||
|
||||
const SESSIONS_DIR = join(os.homedir(), '.openclaw', 'agents', 'main', 'sessions');
|
||||
|
||||
export async function GET() {
|
||||
// Try CLI first
|
||||
try {
|
||||
const { stdout } = await execAsync('openclaw sessions --json');
|
||||
return NextResponse.json({ sessions: JSON.parse(stdout).sessions, source: 'cli' });
|
||||
} catch {}
|
||||
|
||||
// Fallback to file
|
||||
const index = await readFile(join(SESSIONS_DIR, 'sessions.json'), 'utf-8');
|
||||
const sessionsMap = JSON.parse(index);
|
||||
|
||||
const sessions = await Promise.all(
|
||||
Object.entries(sessionsMap).map(async ([key, data]) => ({
|
||||
key,
|
||||
label: getSessionLabel(key),
|
||||
kind: getSessionKind(key),
|
||||
lastActivity: new Date(data.updatedAt).toISOString(),
|
||||
messageCount: await getMessageCount(key),
|
||||
totalTokens: data.totalTokens || 0,
|
||||
estimatedCost: calculateCost(data),
|
||||
}))
|
||||
);
|
||||
|
||||
return NextResponse.json({ sessions, source: 'file' });
|
||||
}
|
||||
```
|
||||
|
||||
### Activity Indicator Component
|
||||
|
||||
```tsx
|
||||
// components/ActivityIndicator.tsx
|
||||
export function ActivityIndicator({ lastActivity }: { lastActivity: Date }) {
|
||||
const now = Date.now();
|
||||
const hourAgo = now - 60 * 60 * 1000;
|
||||
const dayAgo = now - 24 * 60 * 60 * 1000;
|
||||
|
||||
const color = lastActivity.getTime() > hourAgo
|
||||
? 'bg-green-500'
|
||||
: lastActivity.getTime() > dayAgo
|
||||
? 'bg-yellow-500'
|
||||
: 'bg-gray-500';
|
||||
|
||||
const glow = lastActivity.getTime() > hourAgo
|
||||
? 'shadow-[0_0_6px_rgba(34,197,94,0.5)]'
|
||||
: '';
|
||||
|
||||
return (
|
||||
<div className={`w-2 h-2 rounded-full ${color} ${glow}`} />
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Cost Estimation Utility
|
||||
|
||||
```typescript
|
||||
// lib/cost-estimation.ts
|
||||
const RATES = {
|
||||
opus: { input: 15, output: 75 },
|
||||
sonnet: { input: 3, output: 15 },
|
||||
haiku: { input: 0.25, output: 1.25 },
|
||||
};
|
||||
|
||||
export function estimateCost(model: string, inputTokens: number, outputTokens: number): number {
|
||||
const tier = model.includes('opus') ? 'opus'
|
||||
: model.includes('sonnet') ? 'sonnet'
|
||||
: 'haiku';
|
||||
|
||||
const rates = RATES[tier];
|
||||
return (inputTokens / 1_000_000 * rates.input) +
|
||||
(outputTokens / 1_000_000 * rates.output);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Summary
|
||||
|
||||
**Best patterns to steal:**
|
||||
|
||||
1. **Clawtrol's session chat** - Clean two-panel layout with activity dots
|
||||
2. **Mission Control's planning flow** - Multi-choice Q&A with polling
|
||||
3. **Clawtrol's JSONL parsing** - Efficient reverse-iteration for last N messages
|
||||
4. **Mission Control's SSE events** - Simple broadcast pattern with client registry
|
||||
5. **Activity color logic** - Hour = green, day = yellow, older = dim
|
||||
|
||||
**Don't copy:**
|
||||
|
||||
1. Telegram Bot API integration - Use OpenClaw `chat.send` instead
|
||||
2. File-based session index - Mosaic Stack has PostgreSQL
|
||||
3. PM2 daemon management - Use Docker/systemd
|
||||
|
||||
**Next steps:**
|
||||
|
||||
1. Create `/app/(dashboard)/sessions` page with session list
|
||||
2. Add chat view at `/app/(dashboard)/sessions/[key]`
|
||||
3. Wire `/api/sessions` route to OpenClaw CLI or sessions.json
|
||||
4. Add `ActivityIndicator` component to session cards
|
||||
5. Add "Start Planning" button to task cards in Kanban
|
||||
465
docs/research/02-widgets-usage-config-research.md
Normal file
465
docs/research/02-widgets-usage-config-research.md
Normal file
@@ -0,0 +1,465 @@
|
||||
# Widget Layouts + Usage Tracking + Config Management Research
|
||||
|
||||
**Date:** 2026-03-01
|
||||
**Sources:**
|
||||
- [LobsterBoard](https://github.com/Curbob/LobsterBoard) — 50+ drag-and-drop widgets, SSE, layout templates
|
||||
- [VidClaw](https://github.com/madrzak/vidclaw) — Soul/config editor, usage tracking, skills manager
|
||||
|
||||
**Target:** Mosaic Stack (Next.js 15 / React 19 / NestJS / shadcn/ui / PostgreSQL)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
| Feature | LobsterBoard | VidClaw | Mosaic Stack Current | Quick Win? |
|
||||
|---------|--------------|---------|---------------------|------------|
|
||||
| Drag-and-drop widgets | ✅ Full | — | ⚠️ WidgetGrid exists, needs enabling | **Yes (30min)** |
|
||||
| Layout persistence | ✅ JSON to server | — | ✅ API + DB | Done |
|
||||
| SSE real-time | ✅ System stats | — | ✅ Already implemented | Done |
|
||||
| Usage widget (header) | — | ✅ Compact popover | ❌ Full page only | **Yes (30min)** |
|
||||
| Token parsing | — | ✅ JSONL session files | ⚠️ API-based | Low priority |
|
||||
| Soul/config editor | — | ✅ Multi-file + history | ❌ Not in UI | **Yes (1-2h)** |
|
||||
| Skills manager | — | ✅ Full CRUD + toggle | ❌ Not in UI | **Yes (1-2h)** |
|
||||
| Templates | ✅ Layout presets | ✅ Soul templates | ❌ None | Medium |
|
||||
|
||||
---
|
||||
|
||||
## 1. Widget System (LobsterBoard)
|
||||
|
||||
### Widget Registry Pattern
|
||||
|
||||
LobsterBoard uses a global `WIDGETS` object where each widget is self-contained:
|
||||
|
||||
```javascript
|
||||
const WIDGETS = {
|
||||
'weather': {
|
||||
name: 'Local Weather',
|
||||
icon: '🌡️',
|
||||
category: 'small', // 'small' | 'large' | 'layout'
|
||||
description: 'Shows current weather...',
|
||||
defaultWidth: 200,
|
||||
defaultHeight: 120,
|
||||
hasApiKey: false,
|
||||
properties: { // User-configurable defaults
|
||||
title: 'Local Weather',
|
||||
location: 'Atlanta',
|
||||
units: 'F',
|
||||
refreshInterval: 600
|
||||
},
|
||||
preview: `<div>...</div>`,
|
||||
generateHtml: (props) => `...`,
|
||||
generateJs: (props) => `...`
|
||||
},
|
||||
// 50+ more widgets
|
||||
};
|
||||
```
|
||||
|
||||
**Key patterns:**
|
||||
1. **Widget as code generator** — Each widget produces its own HTML + JS at render time
|
||||
2. **Shared SSE** — System stats widgets share one `EventSource('/api/stats/stream')` with a callback registry
|
||||
3. **Edit/View mode toggle** — Widget JS stops in edit mode, resumes in view mode
|
||||
4. **20px grid snapping** — All positions snap to grid during drag
|
||||
5. **Icon theming** — Dual emoji + Phosphor icon map per widget type
|
||||
|
||||
### Layout Persistence Schema
|
||||
|
||||
```json
|
||||
{
|
||||
"canvas": { "width": 1920, "height": 1080 },
|
||||
"fontScale": 1.0,
|
||||
"widgets": [
|
||||
{
|
||||
"id": "widget-1",
|
||||
"type": "weather",
|
||||
"x": 20, "y": 40,
|
||||
"width": 200, "height": 120,
|
||||
"properties": { "title": "Weather", "location": "Kansas City", "units": "F" }
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Saved via `POST /config` with `Content-Type: application/json`. Loaded on startup, starts in view mode.
|
||||
|
||||
### What Mosaic Stack Already Has
|
||||
|
||||
Mosaic's dashboard (`page.tsx`) already has:
|
||||
- ✅ `WidgetGrid` with `react-grid-layout`
|
||||
- ✅ `WidgetPlacement` type in `@mosaic/shared`
|
||||
- ✅ Layout CRUD API (`fetchDefaultLayout`, `createLayout`, `updateLayout`)
|
||||
- ✅ `DEFAULT_LAYOUT` for new users
|
||||
- ✅ Debounced auto-save on layout change (800ms)
|
||||
|
||||
**Gap:** Widget drag-and-drop may need enabling. No dynamic widget registration or per-widget config panel yet.
|
||||
|
||||
### Recommendations
|
||||
|
||||
| Priority | Feature | Effort | Impact |
|
||||
|----------|---------|--------|--------|
|
||||
| 🔴 High | Verify/enable drag-and-drop in WidgetGrid | 30min | Core UX |
|
||||
| 🔴 High | Widget picker modal (add/remove) | 1h | Customization |
|
||||
| 🟡 Med | Per-widget config dialog | 2h | Deeper customization |
|
||||
| 🟢 Low | Layout template presets | 2h | Onboarding |
|
||||
|
||||
---
|
||||
|
||||
## 2. Usage Tracking (VidClaw)
|
||||
|
||||
### Backend: JSONL Session Parsing
|
||||
|
||||
VidClaw's `server/controllers/usage.js` reads OpenClaw session transcript files directly:
|
||||
|
||||
```javascript
|
||||
export function getUsage(req, res) {
|
||||
const sessionsDir = path.join(OPENCLAW_DIR, 'agents', 'main', 'sessions');
|
||||
const tz = getTimezone();
|
||||
const todayStart = startOfDayInTz(now, tz);
|
||||
const weekStart = startOfWeekInTz(now, tz);
|
||||
|
||||
const files = fs.readdirSync(sessionsDir).filter(f => f.endsWith('.jsonl'));
|
||||
for (const file of files) {
|
||||
for (const line of content.split('\n').filter(Boolean)) {
|
||||
const entry = JSON.parse(line);
|
||||
const usage = entry.message?.usage || entry.usage;
|
||||
if (usage?.cost?.total) {
|
||||
const tokens = (usage.input || 0) + (usage.output || 0) + (usage.cacheRead || 0);
|
||||
const cost = usage.cost.total;
|
||||
// Aggregate by day/week/month...
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Also: 5-hour rolling "session" window
|
||||
const SESSION_LIMIT = 45_000_000;
|
||||
const WEEKLY_LIMIT = 180_000_000;
|
||||
|
||||
res.json({
|
||||
model: 'claude-sonnet-4-20250514',
|
||||
tiers: [
|
||||
{ label: 'Current session', percent: 45, resetsIn: '2h 15m', tokens: 20000000, cost: 12.50 },
|
||||
{ label: 'Current week', percent: 32, resetsIn: '4d 8h', tokens: 58000000, cost: 38.20 }
|
||||
],
|
||||
details: {
|
||||
today: { tokens, cost, sessions },
|
||||
week: { tokens, cost, sessions },
|
||||
month: { tokens, cost, sessions }
|
||||
}
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
**Key design choices:**
|
||||
- Multi-tier limits (session 45M + weekly 180M tokens)
|
||||
- Timezone-aware day/week boundaries
|
||||
- Rolling 5-hour session window
|
||||
- Includes cost tracking from `usage.cost.total`
|
||||
|
||||
### Frontend: Compact Header Widget
|
||||
|
||||
VidClaw's `UsageWidget.tsx` is a **popover in the header bar** — not a full page:
|
||||
|
||||
```tsx
|
||||
export default function UsageWidget() {
|
||||
const [expanded, setExpanded] = useState(false);
|
||||
const { data: usage } = useUsage();
|
||||
|
||||
const sessionPct = usage?.tiers?.[0]?.percent ?? 0;
|
||||
const pillColor = sessionPct > 80 ? 'text-red-400' : sessionPct > 60 ? 'text-amber-400' : 'text-emerald-400';
|
||||
|
||||
return (
|
||||
<div className="relative">
|
||||
<button onClick={() => setExpanded(!expanded)}
|
||||
className="flex items-center gap-2 bg-secondary/50 rounded-full px-4 py-1.5 text-xs">
|
||||
<Zap size={12} className="text-orange-400" />
|
||||
<span className="text-muted-foreground">{model}</span>
|
||||
<div className="w-16 h-1.5 bg-secondary rounded-full overflow-hidden">
|
||||
<div className={barColor} style={{ width: `${sessionPct}%` }} />
|
||||
</div>
|
||||
<span className={pillColor}>{sessionPct}%</span>
|
||||
</button>
|
||||
|
||||
{expanded && (
|
||||
<div className="absolute right-0 top-full mt-2 w-80 bg-card border rounded-lg shadow-xl p-4">
|
||||
{/* Model selector */}
|
||||
<select value={model} onChange={switchModel}>...</select>
|
||||
{/* Progress bars per tier */}
|
||||
{tiers.map(tier => <ProgressBar key={tier.label} {...tier} />)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
Color coding: green (<60%), amber (60-80%), red (>80%). Includes model switcher.
|
||||
|
||||
### What Mosaic Stack Has
|
||||
|
||||
Full usage page (430+ lines) with Recharts: line charts, bar charts, pie charts, time range selector. **But no compact header widget.**
|
||||
|
||||
### Recommendations
|
||||
|
||||
| Priority | Feature | Effort | Impact |
|
||||
|----------|---------|--------|--------|
|
||||
| 🔴 High | Compact UsageWidget in header | 30min | Always-visible usage |
|
||||
| 🔴 High | Session + weekly limit % | 1h | Know quota status |
|
||||
| 🟡 Med | Model switcher in popover | 30min | Quick model changes |
|
||||
| 🟢 Low | JSONL parsing backend | 3h | Real-time session tracking |
|
||||
|
||||
---
|
||||
|
||||
## 3. Soul/Config Editor (VidClaw)
|
||||
|
||||
### Backend
|
||||
|
||||
```javascript
|
||||
// server/controllers/soul.js
|
||||
const FILE_TABS = ['SOUL.md', 'IDENTITY.md', 'USER.md', 'AGENTS.md'];
|
||||
|
||||
export function getSoul(req, res) {
|
||||
const content = fs.readFileSync(path.join(WORKSPACE, 'SOUL.md'), 'utf-8');
|
||||
res.json({ content, lastModified: stat.mtime.toISOString() });
|
||||
}
|
||||
|
||||
export function putSoul(req, res) {
|
||||
const old = fs.readFileSync(fp, 'utf-8');
|
||||
if (old) appendHistory(histPath, old); // Auto-version on every save
|
||||
fs.writeFileSync(fp, req.body.content);
|
||||
res.json({ success: true });
|
||||
}
|
||||
|
||||
export function getSoulHistory(req, res) {
|
||||
res.json(readHistoryFile('soul-history.json'));
|
||||
// Returns: [{ content, timestamp }]
|
||||
}
|
||||
|
||||
export function revertSoul(req, res) {
|
||||
appendHistory(histPath, currentContent); // Backup before revert
|
||||
fs.writeFileSync(fp, history[req.body.index].content);
|
||||
res.json({ success: true, content });
|
||||
}
|
||||
```
|
||||
|
||||
### Frontend
|
||||
|
||||
`SoulEditor.tsx` (10KB) — full-featured editor:
|
||||
|
||||
1. **File tabs** — SOUL.md, IDENTITY.md, USER.md, AGENTS.md
|
||||
2. **Code editor** — Textarea with Tab support, Ctrl+S save
|
||||
3. **Right sidebar** with two tabs:
|
||||
- **Templates** — Pre-built soul templates, click to preview, "Use Template" to apply
|
||||
- **History** — Reverse-chronological versions, click to preview, hover to show "Revert"
|
||||
4. **Footer** — Char count, last modified timestamp, dirty indicator, Reset/Save buttons
|
||||
5. **Dirty state** — Yellow dot on tab, "Unsaved changes" warning, confirm before switching tabs
|
||||
|
||||
### Recommendations for Mosaic Stack
|
||||
|
||||
| Priority | Feature | Effort | Impact |
|
||||
|----------|---------|--------|--------|
|
||||
| 🔴 High | Basic editor page with file tabs | 1h | Removes CLI dependency |
|
||||
| 🔴 High | Save + auto-version history | 30min | Safety net for edits |
|
||||
| 🟡 Med | Template sidebar | 1h | Onboarding for new users |
|
||||
| 🟡 Med | Preview before apply/revert | 30min | Prevent mistakes |
|
||||
| 🟢 Low | Syntax highlighting (Monaco) | 1h | Polish |
|
||||
|
||||
**NestJS endpoint sketch:**
|
||||
```typescript
|
||||
@Controller('workspace')
|
||||
export class WorkspaceController {
|
||||
@Get('file')
|
||||
getFile(@Query('name') name: string) {
|
||||
// Validate name is in allowed list
|
||||
// Read from workspace dir, return { content, lastModified }
|
||||
}
|
||||
|
||||
@Put('file')
|
||||
putFile(@Query('name') name: string, @Body() body: { content: string }) {
|
||||
// Append old content to history JSON
|
||||
// Write new content
|
||||
}
|
||||
|
||||
@Get('file/history')
|
||||
getHistory(@Query('name') name: string) {
|
||||
// Return history entries
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Skills Manager (VidClaw)
|
||||
|
||||
### Backend: Skill Scanning
|
||||
|
||||
`server/lib/skills.js` scans multiple directories for skills:
|
||||
|
||||
```javascript
|
||||
const SKILL_SCAN_DIRS = {
|
||||
bundled: ['/opt/openclaw/skills'],
|
||||
managed: ['~/.config/mosaic/skills'],
|
||||
workspace: ['~/.openclaw/workspace/skills']
|
||||
};
|
||||
|
||||
export function scanSkills() {
|
||||
const config = readOpenclawJson();
|
||||
const entries = config.skills?.entries || {}; // Enabled/disabled state
|
||||
|
||||
for (const [source, roots] of Object.entries(SKILL_SCAN_DIRS)) {
|
||||
for (const d of fs.readdirSync(rootDir, { withFileTypes: true })) {
|
||||
const content = fs.readFileSync(path.join(d.name, 'SKILL.md'), 'utf-8');
|
||||
const fm = parseFrontmatter(content); // Parse YAML frontmatter
|
||||
|
||||
skills.push({
|
||||
id: d.name,
|
||||
name: fm.name || d.name,
|
||||
description: fm.description || '',
|
||||
source, // 'bundled' | 'managed' | 'workspace'
|
||||
enabled: entries[id]?.enabled ?? true,
|
||||
path: skillPath,
|
||||
});
|
||||
}
|
||||
}
|
||||
return skills;
|
||||
}
|
||||
```
|
||||
|
||||
### Backend: CRUD
|
||||
|
||||
```javascript
|
||||
// Toggle: writes to openclaw.json config
|
||||
export function toggleSkill(req, res) {
|
||||
config.skills.entries[id] = { enabled: !current };
|
||||
writeOpenclawJson(config);
|
||||
}
|
||||
|
||||
// Create: writes SKILL.md with frontmatter
|
||||
export function createSkill(req, res) {
|
||||
const dir = path.join(SKILLS_DIRS.workspace, name);
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
fs.writeFileSync(path.join(dir, 'SKILL.md'),
|
||||
`---\nname: ${name}\ndescription: ${desc}\n---\n\n${instructions}`);
|
||||
}
|
||||
|
||||
// Delete: workspace skills only
|
||||
export function deleteSkill(req, res) {
|
||||
if (skill.source !== 'workspace') return res.status(403);
|
||||
fs.rmSync(skill.path, { recursive: true });
|
||||
}
|
||||
```
|
||||
|
||||
### Frontend
|
||||
|
||||
`SkillsManager.tsx` (12KB):
|
||||
|
||||
1. **Stats cards** — Total, Enabled, Bundled, Workspace counts
|
||||
2. **Filters** — Search, source filter dropdown, status filter dropdown
|
||||
3. **Skill cards** — Name + source badge + toggle switch + expand/collapse
|
||||
4. **Expanded view** — Shows full SKILL.md content (lazy-loaded)
|
||||
5. **Create modal** — Name (slug), description, instructions (markdown textarea)
|
||||
6. **Source badges** — Color-coded: blue=bundled, orange=managed, green=workspace
|
||||
7. **Delete** — Only workspace skills, with confirmation
|
||||
|
||||
### Recommendations for Mosaic Stack
|
||||
|
||||
| Priority | Feature | Effort | Impact |
|
||||
|----------|---------|--------|--------|
|
||||
| 🔴 High | Skills list with toggle | 1h | Visibility + control |
|
||||
| 🟡 Med | Create skill modal | 1h | No CLI needed |
|
||||
| 🟡 Med | Skill content viewer | 30min | See what skills do |
|
||||
| 🟢 Low | Search + filters | 30min | Polish for 100+ skills |
|
||||
|
||||
---
|
||||
|
||||
## 5. Quick Wins — Prioritized Implementation Plan
|
||||
|
||||
### 🚀 #1: Compact Usage Widget in Header (30 min)
|
||||
- Create `components/UsageWidget.tsx` using shadcn `Popover` + `Progress`
|
||||
- Reuse existing `useUsageSummary` hook
|
||||
- Add to authenticated layout header
|
||||
- Color-code: green/amber/red based on percentage
|
||||
|
||||
### 🚀 #2: Enable Widget Drag-and-Drop (30 min)
|
||||
- Check `WidgetGrid` for `isDraggable`/`static` props
|
||||
- Enable drag + resize in react-grid-layout
|
||||
- Verify auto-save still works after moves
|
||||
|
||||
### 🚀 #3: Soul Editor Page (1-2h)
|
||||
- New page: `settings/soul/page.tsx`
|
||||
- File tabs: SOUL.md, IDENTITY.md, USER.md, AGENTS.md
|
||||
- Backend: `GET/PUT /api/workspace/file?name=SOUL.md`
|
||||
- Auto-version history on save
|
||||
- Simple Textarea with Save button
|
||||
|
||||
### 🚀 #4: Skills List + Toggle (1-2h)
|
||||
- New page: `settings/skills/page.tsx`
|
||||
- Backend: `GET /api/skills`, `POST /api/skills/:id/toggle`
|
||||
- Scan skill directories, parse frontmatter
|
||||
- Toggle switch per skill using shadcn `Switch`
|
||||
|
||||
### 🚀 #5: Dashboard Empty State (30 min)
|
||||
- Show "Add your first widget" card when layout is empty
|
||||
- Link to widget picker
|
||||
|
||||
**Total estimated effort for all 5: ~4-5 hours for a dramatically more complete UI.**
|
||||
|
||||
---
|
||||
|
||||
## 6. Schemas Worth Borrowing
|
||||
|
||||
### Skill Type (for Mosaic Stack shared package)
|
||||
```typescript
|
||||
interface Skill {
|
||||
id: string;
|
||||
name: string;
|
||||
description: string;
|
||||
source: 'bundled' | 'managed' | 'workspace';
|
||||
enabled: boolean;
|
||||
path: string;
|
||||
}
|
||||
```
|
||||
|
||||
### Usage Tier Type
|
||||
```typescript
|
||||
interface UsageTier {
|
||||
label: string;
|
||||
percent: number;
|
||||
resetsIn: string;
|
||||
tokens: number;
|
||||
cost: number;
|
||||
}
|
||||
```
|
||||
|
||||
### Widget Definition Type (if building registry)
|
||||
```typescript
|
||||
interface WidgetDefinition {
|
||||
id: string;
|
||||
name: string;
|
||||
icon: string;
|
||||
category: 'kpi' | 'chart' | 'list' | 'system';
|
||||
description: string;
|
||||
defaultSize: { w: number; h: number };
|
||||
configSchema?: Record<string, { type: string; label: string; default: unknown }>;
|
||||
component: React.ComponentType<WidgetProps>;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key File References
|
||||
|
||||
### LobsterBoard
|
||||
- `js/widgets.js` — 50+ widget definitions with HTML/JS generators
|
||||
- `js/builder.js` — Canvas, drag-drop, resize, edit/view mode, config save/load
|
||||
|
||||
### VidClaw
|
||||
- `server/controllers/usage.js` — JSONL token parsing, multi-tier limits
|
||||
- `server/controllers/soul.js` — SOUL.md CRUD + version history
|
||||
- `server/controllers/skills.js` — Skills CRUD (toggle, create, delete)
|
||||
- `server/lib/skills.js` — Directory scanning + frontmatter parsing
|
||||
- `src/components/Usage/UsageWidget.tsx` — Compact header usage popover
|
||||
- `src/components/Soul/SoulEditor.tsx` — Multi-file editor with history + templates
|
||||
- `src/components/Skills/SkillsManager.tsx` — Skills list, filter, toggle, create
|
||||
|
||||
---
|
||||
|
||||
*Research completed 2026-03-01 by subagent for Mosaic Stack development.*
|
||||
163
docs/research/03-security-fleet-synthesis.md
Normal file
163
docs/research/03-security-fleet-synthesis.md
Normal file
@@ -0,0 +1,163 @@
|
||||
# Security Patterns, Lightweight Monitors & Final 10% Synthesis
|
||||
|
||||
**Research Date:** 2026-03-01
|
||||
**Repositories Analyzed:**
|
||||
1. [tugcantopaloglu/openclaw-dashboard](https://github.com/tugcantopaloglu/openclaw-dashboard) — Security-hardened: TOTP MFA, PBKDF2, rate limiting, memory viewer, cron manager
|
||||
2. [Temaki-AI/clawd-control](https://github.com/Temaki-AI/clawd-control) — Lightweight fleet monitor, auto-discovery, agent creation wizard
|
||||
3. [spleck/claw-dashboard](https://github.com/spleck/claw-dashboard) — Terminal-style monitor, btop-inspired
|
||||
4. [23blocks-OS/ai-maestro](https://github.com/23blocks-OS/ai-maestro) — Agent-to-agent messaging, AMP protocol, multi-machine mesh
|
||||
|
||||
---
|
||||
|
||||
## 1. Memory/File Viewer (openclaw-dashboard)
|
||||
|
||||
**How it works:** Reads workspace files directly from filesystem — MEMORY.md, HEARTBEAT.md, memory/YYYY-MM-DD.md. Two API endpoints: `GET /api/memory-files` (list) and `GET /api/memory-file?path=<path>` (read content). Frontend is a simple file browser + markdown viewer. Edits create `.bak` backup files automatically.
|
||||
|
||||
**Security:** Path traversal protection validates all paths stay within workspace root. Read-only by default; edit requires explicit action.
|
||||
|
||||
**Simplest implementation for Mosaic Stack:**
|
||||
- NestJS controller with 2 endpoints (list files, read file)
|
||||
- Path validation middleware (resolve path, check it starts with workspace root)
|
||||
- Next.js page: left sidebar file tree + right panel markdown render
|
||||
- Use `react-markdown` for rendering (already likely in deps)
|
||||
- **Effort: 1-2h**
|
||||
|
||||
---
|
||||
|
||||
## 2. Cron Job Management UI (openclaw-dashboard)
|
||||
|
||||
**How it works:** Reads cron jobs from `$OPENCLAW_DIR/cron/jobs.json`. Three endpoints:
|
||||
- `GET /api/crons` — list all jobs with status
|
||||
- `POST /api/cron/:id/toggle` — enable/disable
|
||||
- `POST /api/cron/:id/run` — manually trigger
|
||||
|
||||
Frontend: table with Name | Schedule | Status | Last Run | Actions columns. Toggle switches and "Run Now" buttons.
|
||||
|
||||
**For Mosaic Stack:** Could be a Settings sub-tab ("Automation"). Back-end reads from DB or config file. NestJS `@nestjs/schedule` already supports cron — just need UI visibility into what's scheduled.
|
||||
|
||||
**Effort: 2-3h**
|
||||
|
||||
---
|
||||
|
||||
## 3. Agent Creation Wizard (clawd-control)
|
||||
|
||||
**How it works:** Guided multi-step form at `create.html`. Agent config fields:
|
||||
```json
|
||||
{
|
||||
"id": "my-agent",
|
||||
"gatewayAgentId": "main",
|
||||
"name": "My Agent",
|
||||
"emoji": "🤖",
|
||||
"host": "127.0.0.1",
|
||||
"port": 18789,
|
||||
"token": "YOUR_GATEWAY_TOKEN",
|
||||
"workspace": "/path/to/agent/workspace"
|
||||
}
|
||||
```
|
||||
|
||||
Backend provisioning logic in `create-agent.mjs`. Auto-discovery via `discover.mjs` finds local agents automatically.
|
||||
|
||||
**For Mosaic Stack:** Already has agents table in DB. Add a "Create Agent" dialog/wizard with: name, type/model, emoji, connection details, workspace path. Multi-step or single form — single form is faster to build.
|
||||
|
||||
**Effort: 2-4h**
|
||||
|
||||
---
|
||||
|
||||
## 4. Fleet Overview UX (all dashboards)
|
||||
|
||||
**What good looks like:**
|
||||
|
||||
| Dashboard | Approach | Key Insight |
|
||||
|-----------|----------|-------------|
|
||||
| clawd-control | Grid of agent cards, single-screen | "See all agents at a glance with health indicators" |
|
||||
| openclaw-dashboard | Sidebar + tabs, sparklines, heatmaps | Rich metrics: sessions, costs, rate limits |
|
||||
| claw-dashboard | Terminal btop-style, 2s refresh | Lightweight, resource-efficient |
|
||||
| ai-maestro | Tree view with auto-coloring | `project-backend-api` → 3-level tree |
|
||||
|
||||
**Key metrics that matter:**
|
||||
- Status indicator (online/offline/error) — most important
|
||||
- Last activity timestamp
|
||||
- Active session count
|
||||
- Token usage / cost
|
||||
- CPU/RAM (if host-level monitoring)
|
||||
- Error count (last 24h)
|
||||
|
||||
**Recommended for Mosaic Stack:** Card grid layout. Each card: emoji + name, colored status dot, last activity time, token count. Click to expand/detail. Add a "Recent Activity" feed below the grid.
|
||||
|
||||
**Effort: 3-4h**
|
||||
|
||||
---
|
||||
|
||||
## 5. AMP Protocol (ai-maestro)
|
||||
|
||||
**What it is:** Agent Messaging Protocol — email-like communication between agents. Priority levels, message types, cryptographic signatures, push notifications. Full spec at agentmessaging.org.
|
||||
|
||||
**Key concept:** "I was the human mailman between 35 agents. AMP removes the human bottleneck."
|
||||
|
||||
**Worth borrowing for Mosaic Stack:**
|
||||
- Simple agent-to-agent message table in PostgreSQL (already have DB)
|
||||
- Priority levels (low/normal/high)
|
||||
- Message types (task/notification/query)
|
||||
- Thread awareness (threadId field)
|
||||
|
||||
**NOT worth borrowing (yet):**
|
||||
- Cryptographic signatures (overkill)
|
||||
- Multi-machine mesh (premature)
|
||||
- Full AMP protocol compliance (too complex)
|
||||
|
||||
**Simple alternative:** Add a `messages` table to Prisma schema with fromAgentId, toAgentId, type, priority, subject, body, threadId, readAt. Poll or WebSocket for delivery. **Effort: 4-8h**
|
||||
|
||||
---
|
||||
|
||||
## 6. Security Patterns Worth Adopting
|
||||
|
||||
**From openclaw-dashboard (already mature in Mosaic Stack):**
|
||||
|
||||
| Pattern | openclaw-dashboard | Mosaic Stack Status | Action |
|
||||
|---------|-------------------|-------------------|--------|
|
||||
| Password hashing | PBKDF2, 100k iterations | Better Auth handles this | ✅ Done |
|
||||
| CSRF protection | N/A (session-based) | Better Auth CSRF | ✅ Done |
|
||||
| RBAC | N/A | Full RBAC implemented | ✅ Done |
|
||||
| Rate limiting | 5 fail → 15min lockout | Not implemented | Add NestJS throttler |
|
||||
| TOTP MFA | Google Auth compatible | Not implemented | P2 — Better Auth plugin exists |
|
||||
| Audit logging | All auth events logged | Not implemented | Add NestJS middleware |
|
||||
| Security headers | HSTS, CSP, X-Frame | Partial | Add helmet middleware |
|
||||
|
||||
**Quick wins:**
|
||||
- `@nestjs/throttler` for rate limiting (30min)
|
||||
- `helmet` middleware for security headers (15min)
|
||||
- Audit log table + middleware (1-2h)
|
||||
|
||||
---
|
||||
|
||||
## 7. Real-Time Updates Pattern
|
||||
|
||||
All four dashboards use real-time updates differently:
|
||||
- openclaw-dashboard: SSE (`/api/live`)
|
||||
- clawd-control: SSE
|
||||
- claw-dashboard: Polling (2s interval)
|
||||
- ai-maestro: WebSocket
|
||||
|
||||
**For Mosaic Stack:** Already has WebSocket for terminal. Use SSE for fleet status (simpler than WebSocket, one-directional is fine). Polling for non-critical pages.
|
||||
|
||||
---
|
||||
|
||||
## Feature Comparison Matrix
|
||||
|
||||
| Feature | openclaw-dash | clawd-control | claw-dash | ai-maestro | Mosaic Stack |
|
||||
|---------|:---:|:---:|:---:|:---:|:---:|
|
||||
| Session mgmt | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| Memory viewer | ✅ | ❌ | ❌ | ✅ | ❌ |
|
||||
| Cron mgmt | ✅ | ❌ | ❌ | ❌ | ❌ |
|
||||
| Agent wizard | ❌ | ✅ | ❌ | ✅ | ❌ |
|
||||
| Fleet overview | ✅ | ✅ | ❌ | ✅ | Partial |
|
||||
| Multi-machine | ❌ | ❌ | ❌ | ✅ | ❌ |
|
||||
| Agent messaging | ❌ | ❌ | ❌ | ✅ | ❌ |
|
||||
| Rate limiting | ✅ | ✅ | ❌ | ❌ | ❌ |
|
||||
| TOTP MFA | ✅ | ❌ | ❌ | ❌ | ❌ |
|
||||
| Real-time | SSE | SSE | Poll | WS | WS (terminal) |
|
||||
| Cost tracking | ✅ | ❌ | ❌ | ❌ | ✅ (usage) |
|
||||
| Terminal UI | ❌ | ❌ | ✅ | ❌ | ✅ (xterm.js) |
|
||||
| Kanban | ❌ | ❌ | ❌ | ✅ | ✅ |
|
||||
| Auth | PBKDF2+MFA | Password | None | N/A | Better Auth |
|
||||
| RBAC | ❌ | ❌ | ❌ | ❌ | ✅ |
|
||||
Reference in New Issue
Block a user