mosaic/stack
1
0
Fork 0
Code Issues 10 Pull Requests Actions Packages Projects Releases 2 Wiki Activity
Files
9a7673bea206ade439728c4fbcbccdd5d6874f9e
stack/docs/design/knowledge-module-issues.md
Jason Woltje 12abdfe81d feat(#93): implement agent spawn via federation 
Implements FED-010: Agent Spawn via Federation feature that enables
spawning and managing Claude agents on remote federated Mosaic Stack
instances via COMMAND message type.

Features:
- Federation agent command types (spawn, status, kill)
- FederationAgentService for handling agent operations
- Integration with orchestrator's agent spawner/lifecycle services
- API endpoints for spawning, querying status, and killing agents
- Full command routing through federation COMMAND infrastructure
- Comprehensive test coverage (12/12 tests passing)

Architecture:
- Hub → Spoke: Spawn agents on remote instances
- Command flow: FederationController → FederationAgentService →
  CommandService → Remote Orchestrator
- Response handling: Remote orchestrator returns agent status/results
- Security: Connection validation, signature verification

Files created:
- apps/api/src/federation/types/federation-agent.types.ts
- apps/api/src/federation/federation-agent.service.ts
- apps/api/src/federation/federation-agent.service.spec.ts

Files modified:
- apps/api/src/federation/command.service.ts (agent command routing)
- apps/api/src/federation/federation.controller.ts (agent endpoints)
- apps/api/src/federation/federation.module.ts (service registration)
- apps/orchestrator/src/api/agents/agents.controller.ts (status endpoint)
- apps/orchestrator/src/api/agents/agents.module.ts (lifecycle integration)

Testing:
- 12/12 tests passing for FederationAgentService
- All command service tests passing
- TypeScript compilation successful
- Linting passed

Refs #93

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-03 14:37:06 -06:00

711 lines
16 KiB
Markdown
Raw Blame History

# Knowledge Module - Implementation Issues
> **Epic:** Knowledge Module
> **Design Doc:** [knowledge-module.md](./knowledge-module.md)
> **Target:** 6 weeks
> **Created:** 2025-01-29
---
## Epic Overview
Build a native knowledge management module for Mosaic Stack with wiki-style linking, semantic search, and graph visualization.
**Labels:** `epic`, `feature`, `knowledge-module`
---
## Phase 1: Foundation (Week 1-2)
### KNOW-001: Database Schema for Knowledge Module
**Priority:** P0
**Estimate:** 4h
**Labels:** `database`, `schema`, `phase-1`
**Description:**
Create Prisma schema and migrations for the Knowledge module.
**Acceptance Criteria:**
- [ ] `KnowledgeEntry` model with all fields
- [ ] `KnowledgeEntryVersion` model for history
- [ ] `KnowledgeLink` model for wiki-links
- [ ] `KnowledgeTag` and `KnowledgeEntryTag` models
- [ ] `KnowledgeEmbedding` model (pgvector ready)
- [ ] All indexes defined
- [ ] Migration runs cleanly
- [ ] Seed data for testing
**Technical Notes:**
- Reference design doc for full schema
- Ensure `@@unique([workspaceId, slug])` constraint
- Add `search_vector` column for full-text search
- pgvector extension may need separate setup
---
### KNOW-002: Entry CRUD API Endpoints
**Priority:** P0
**Estimate:** 6h
**Labels:** `api`, `phase-1`
**Description:**
Implement RESTful API for knowledge entry management.
**Acceptance Criteria:**
- [ ] `POST /api/knowledge/entries` - Create entry
- [ ] `GET /api/knowledge/entries` - List entries (paginated, filterable)
- [ ] `GET /api/knowledge/entries/:slug` - Get single entry
- [ ] `PUT /api/knowledge/entries/:slug` - Update entry
- [ ] `DELETE /api/knowledge/entries/:slug` - Archive entry (soft delete)
- [ ] Workspace isolation enforced
- [ ] Input validation with class-validator
- [ ] OpenAPI/Swagger documentation
**Technical Notes:**
- Follow existing Mosaic API patterns
- Use `@WorkspaceGuard()` for tenant isolation
- Slug generation from title with collision handling
---
### KNOW-003: Tag Management API
**Priority:** P0
**Estimate:** 3h
**Labels:** `api`, `phase-1`
**Description:**
Implement tag CRUD and entry-tag associations.
**Acceptance Criteria:**
- [ ] `GET /api/knowledge/tags` - List workspace tags
- [ ] `POST /api/knowledge/tags` - Create tag
- [ ] `PUT /api/knowledge/tags/:slug` - Update tag
- [ ] `DELETE /api/knowledge/tags/:slug` - Delete tag
- [ ] `GET /api/knowledge/tags/:slug/entries` - Entries with tag
- [ ] Entry creation/update accepts tag slugs
- [ ] Auto-create tags if `autoCreateTags: true`
---
### KNOW-004: Basic Markdown Rendering
**Priority:** P0
**Estimate:** 4h
**Labels:** `api`, `rendering`, `phase-1`
**Description:**
Render markdown content to HTML with caching.
**Acceptance Criteria:**
- [ ] Markdown-to-HTML conversion on entry save
- [ ] Support GFM (tables, task lists, strikethrough)
- [ ] Code syntax highlighting (highlight.js or Shiki)
- [ ] Sanitize HTML output (XSS prevention)
- [ ] Cache rendered HTML in `contentHtml` field
- [ ] Invalidate cache on content update
**Technical Notes:**
- Use `marked` or `remark` for parsing
- Wiki-links (`[[...]]`) parsed but not resolved yet (Phase 2)
---
### KNOW-005: Entry List Page UI
**Priority:** P0
**Estimate:** 6h
**Labels:** `frontend`, `phase-1`
**Description:**
Build the knowledge entry list page in the web UI.
**Acceptance Criteria:**
- [ ] List view with title, summary, tags, updated date
- [ ] Filter by status (draft/published/archived)
- [ ] Filter by tag
- [ ] Sort by updated/created/title
- [ ] Pagination
- [ ] Quick search (client-side filter)
- [ ] Create new entry button → editor
- [ ] Responsive design
---
### KNOW-006: Entry Detail/Editor Page UI
**Priority:** P0
**Estimate:** 8h
**Labels:** `frontend`, `editor`, `phase-1`
**Description:**
Build the entry view and edit page.
**Acceptance Criteria:**
- [ ] View mode with rendered markdown
- [ ] Edit mode with markdown editor
- [ ] Split view option (edit + preview)
- [ ] Title editing
- [ ] Tag selection/creation
- [ ] Status dropdown
- [ ] Save/cancel/delete actions
- [ ] Unsaved changes warning
- [ ] Keyboard shortcuts (Cmd+S to save)
**Technical Notes:**
- Consider CodeMirror or Monaco for editor
- May use existing rich-text patterns from Mosaic
---
## Phase 2: Linking (Week 2-3)
### KNOW-007: Wiki-Link Parser
**Priority:** P0
**Estimate:** 4h
**Labels:** `api`, `parsing`, `phase-2`
**Description:**
Parse `[[wiki-link]]` syntax from markdown content.
**Acceptance Criteria:**
- [ ] Extract all `[[...]]` patterns from content
- [ ] Support `[[slug]]` basic syntax
- [ ] Support `[[slug|display text]]` aliased links
- [ ] Support `[[slug#header]]` section links
- [ ] Return structured link data with positions
- [ ] Handle edge cases (nested brackets, escaping)
**Technical Notes:**
```typescript
interface ParsedLink {
raw: string; // "[[design|Design Doc]]"
target: string; // "design"
display: string; // "Design Doc"
section?: string; // "header" if [[design#header]]
position: { start: number; end: number };
}
```
---
### KNOW-008: Link Resolution Service
**Priority:** P0
**Estimate:** 4h
**Labels:** `api`, `phase-2`
**Description:**
Resolve parsed wiki-links to actual entries.
**Acceptance Criteria:**
- [ ] Resolve by exact slug match
- [ ] Resolve by title match (case-insensitive)
- [ ] Fuzzy match fallback (optional)
- [ ] Mark unresolved links as broken
- [ ] Extract surrounding context for link record
- [ ] Batch resolution for efficiency
---
### KNOW-009: Link Storage and Sync
**Priority:** P0
**Estimate:** 4h
**Labels:** `api`, `database`, `phase-2`
**Description:**
Store links in database and keep in sync with content.
**Acceptance Criteria:**
- [ ] On entry save: parse → resolve → store links
- [ ] Remove stale links on update
- [ ] `GET /api/knowledge/entries/:slug/links/outgoing`
- [ ] `GET /api/knowledge/entries/:slug/links/incoming` (backlinks)
- [ ] `GET /api/knowledge/entries/:slug/links/broken`
- [ ] Broken link report for workspace
---
### KNOW-010: Backlinks Display UI
**Priority:** P1
**Estimate:** 3h
**Labels:** `frontend`, `phase-2`
**Description:**
Show incoming links (backlinks) on entry pages.
**Acceptance Criteria:**
- [ ] Backlinks section on entry detail page
- [ ] Show linking entry title + context snippet
- [ ] Click to navigate to linking entry
- [ ] Count badge in sidebar/header
- [ ] Empty state when no backlinks
---
### KNOW-011: Link Autocomplete in Editor
**Priority:** P1
**Estimate:** 6h
**Labels:** `frontend`, `editor`, `phase-2`
**Description:**
Autocomplete suggestions when typing `[[`.
**Acceptance Criteria:**
- [ ] Trigger on `[[` typed in editor
- [ ] Show dropdown with matching entries
- [ ] Search by title and slug
- [ ] Show recent entries first, then alphabetical
- [ ] Insert selected entry as `[[slug]]` or `[[slug|title]]`
- [ ] Keyboard navigation (arrows, enter, escape)
- [ ] Debounced API calls
---
### KNOW-012: Render Links in View Mode
**Priority:** P0
**Estimate:** 3h
**Labels:** `frontend`, `rendering`, `phase-2`
**Description:**
Render wiki-links as clickable links in entry view.
**Acceptance Criteria:**
- [ ] `[[slug]]` renders as link to `/knowledge/slug`
- [ ] `[[slug|text]]` shows custom text
- [ ] Broken links styled differently (red, dashed underline)
- [ ] Hover preview (optional, stretch goal)
---
## Phase 3: Search (Week 3-4)
### KNOW-013: Full-Text Search Setup
**Priority:** P0
**Estimate:** 4h
**Labels:** `database`, `search`, `phase-3`
**Description:**
Set up PostgreSQL full-text search for entries.
**Acceptance Criteria:**
- [ ] Add `tsvector` column to entries table
- [ ] Create GIN index on search vector
- [ ] Weight title (A), summary (B), content (C)
- [ ] Trigger to update vector on insert/update
- [ ] Verify search performance with test data
---
### KNOW-014: Search API Endpoint
**Priority:** P0
**Estimate:** 4h
**Labels:** `api`, `search`, `phase-3`
**Description:**
Implement search API with full-text search.
**Acceptance Criteria:**
- [ ] `GET /api/knowledge/search?q=...`
- [ ] Return ranked results with snippets
- [ ] Highlight matching terms in snippets
- [ ] Filter by tags, status
- [ ] Pagination
- [ ] Response time < 200ms
---
### KNOW-015: Search UI
**Priority:** P0
**Estimate:** 6h
**Labels:** `frontend`, `search`, `phase-3`
**Description:**
Build search interface in web UI.
**Acceptance Criteria:**
- [ ] Search input in knowledge module header
- [ ] Search results page
- [ ] Highlighted snippets
- [ ] Filter sidebar (tags, status)
- [ ] "No results" state with suggestions
- [ ] Search as you type (debounced)
- [ ] Keyboard shortcut (Cmd+K) to focus search
---
### KNOW-016: pgvector Setup
**Priority:** P1
**Estimate:** 4h
**Labels:** `database`, `vector`, `phase-3`
**Description:**
Set up pgvector extension for semantic search.
**Acceptance Criteria:**
- [ ] Enable pgvector extension in PostgreSQL
- [ ] Create embeddings table with vector column
- [ ] HNSW index for fast similarity search
- [ ] Verify extension works in dev and prod
**Technical Notes:**
- May need PostgreSQL 15+ for best pgvector support
- Consider managed options (Supabase, Neon) if self-hosting is complex
---
### KNOW-017: Embedding Generation Pipeline
**Priority:** P1
**Estimate:** 6h
**Labels:** `api`, `vector`, `phase-3`
**Description:**
Generate embeddings for entries using OpenAI or local model.
**Acceptance Criteria:**
- [ ] Service to generate embeddings from text
- [ ] On entry create/update: queue embedding job
- [ ] Background worker processes queue
- [ ] Store embedding in database
- [ ] Handle API rate limits and errors
- [ ] Config for embedding model selection
**Technical Notes:**
- Start with OpenAI `text-embedding-ada-002`
- Consider local options (sentence-transformers) for cost/privacy
---
### KNOW-018: Semantic Search API
**Priority:** P1
**Estimate:** 4h
**Labels:** `api`, `search`, `vector`, `phase-3`
**Description:**
Implement semantic (vector) search endpoint.
**Acceptance Criteria:**
- [ ] `POST /api/knowledge/search/semantic`
- [ ] Accept natural language query
- [ ] Generate query embedding
- [ ] Find similar entries by cosine similarity
- [ ] Return results with similarity scores
- [ ] Configurable similarity threshold
---
## Phase 4: Graph (Week 4-5)
### KNOW-019: Graph Data API
**Priority:** P1
**Estimate:** 4h
**Labels:** `api`, `graph`, `phase-4`
**Description:**
API to retrieve knowledge graph data.
**Acceptance Criteria:**
- [ ] `GET /api/knowledge/graph` - Full graph (nodes + edges)
- [ ] `GET /api/knowledge/graph/:slug` - Subgraph centered on entry
- [ ] `GET /api/knowledge/graph/stats` - Graph statistics
- [ ] Include orphan detection
- [ ] Filter by tag, status
- [ ] Limit node count option
---
### KNOW-020: Graph Visualization Component
**Priority:** P1
**Estimate:** 8h
**Labels:** `frontend`, `graph`, `phase-4`
**Description:**
Interactive knowledge graph visualization.
**Acceptance Criteria:**
- [ ] Force-directed graph layout
- [ ] Nodes sized by connection count
- [ ] Nodes colored by status
- [ ] Click node to navigate or show details
- [ ] Zoom and pan controls
- [ ] Layout toggle (force, hierarchical, radial)
- [ ] Performance OK with 500+ nodes
**Technical Notes:**
- Use D3.js or Cytoscape.js
- Consider WebGL renderer for large graphs
---
### KNOW-021: Entry-Centered Graph View
**Priority:** P2
**Estimate:** 4h
**Labels:** `frontend`, `graph`, `phase-4`
**Description:**
Show mini-graph on entry detail page.
**Acceptance Criteria:**
- [ ] Small graph showing entry + direct connections
- [ ] 1-2 hop neighbors
- [ ] Click to expand or navigate
- [ ] Toggle to show/hide on entry page
---
### KNOW-022: Graph Statistics Dashboard
**Priority:** P2
**Estimate:** 3h
**Labels:** `frontend`, `graph`, `phase-4`
**Description:**
Dashboard showing knowledge base health.
**Acceptance Criteria:**
- [ ] Total entries, links, tags
- [ ] Orphan entry count (no links)
- [ ] Broken link count
- [ ] Average connections per entry
- [ ] Most connected entries
- [ ] Recently updated
---
## Phase 5: Polish (Week 5-6)
### KNOW-023: Version History API
**Priority:** P1
**Estimate:** 4h
**Labels:** `api`, `versioning`, `phase-5`
**Description:**
API for entry version history.
**Acceptance Criteria:**
- [ ] Create version on each save
- [ ] `GET /api/knowledge/entries/:slug/versions`
- [ ] `GET /api/knowledge/entries/:slug/versions/:v`
- [ ] `POST /api/knowledge/entries/:slug/restore/:v`
- [ ] Version limit per entry (configurable, default 50)
- [ ] Prune old versions
---
### KNOW-024: Version History UI
**Priority:** P1
**Estimate:** 6h
**Labels:** `frontend`, `versioning`, `phase-5`
**Description:**
UI to browse and restore versions.
**Acceptance Criteria:**
- [ ] Version list sidebar/panel
- [ ] Show version date, author, change note
- [ ] Click to view historical version
- [ ] Diff view between versions
- [ ] Restore button with confirmation
- [ ] Compare any two versions
**Technical Notes:**
- Use diff library for content comparison
- Highlight additions/deletions
---
### KNOW-025: Markdown Import
**Priority:** P2
**Estimate:** 4h
**Labels:** `api`, `import`, `phase-5`
**Description:**
Import existing markdown files into knowledge base.
**Acceptance Criteria:**
- [ ] Upload `.md` file(s)
- [ ] Parse frontmatter for metadata
- [ ] Generate slug from filename or title
- [ ] Resolve wiki-links if target entries exist
- [ ] Report on import results
- [ ] Bulk import from folder/zip
---
### KNOW-026: Export Functionality
**Priority:** P2
**Estimate:** 3h
**Labels:** `api`, `export`, `phase-5`
**Description:**
Export entries to markdown/PDF.
**Acceptance Criteria:**
- [ ] Export single entry as markdown
- [ ] Export single entry as PDF
- [ ] Bulk export (all or filtered)
- [ ] Include frontmatter in markdown export
- [ ] Preserve wiki-links in export
---
### KNOW-027: Caching Layer
**Priority:** P1
**Estimate:** 4h
**Labels:** `api`, `performance`, `phase-5`
**Description:**
Implement Valkey caching for knowledge module.
**Acceptance Criteria:**
- [ ] Cache entry JSON
- [ ] Cache rendered HTML
- [ ] Cache graph data
- [ ] Cache search results (short TTL)
- [ ] Proper invalidation on updates
- [ ] Cache hit metrics
---
### KNOW-028: Documentation
**Priority:** P1
**Estimate:** 4h
**Labels:** `docs`, `phase-5`
**Description:**
Document the knowledge module.
**Acceptance Criteria:**
- [ ] User guide for knowledge module
- [ ] API reference (OpenAPI already in place)
- [ ] Wiki-link syntax reference
- [ ] Admin/config documentation
- [ ] Architecture overview for developers
---
## Stretch Goals / Future
### KNOW-029: Real-Time Collaboration
**Priority:** P3
**Labels:** `future`, `collaboration`
**Description:**
Multiple users editing same entry simultaneously.
**Notes:**
- Would require CRDT or OT implementation
- Significant complexity
- Evaluate need before committing
---
### KNOW-030: Entry Templates
**Priority:** P3
**Labels:** `future`, `templates`
**Description:**
Pre-defined templates for common entry types.
**Notes:**
- ADR template
- Design doc template
- Meeting notes template
- Custom templates per workspace
---
### KNOW-031: Attachments
**Priority:** P3
**Labels:** `future`, `attachments`
**Description:**
Upload and embed images/files in entries.
**Notes:**
- S3/compatible storage backend
- Image optimization
- Paste images into editor
---
## Summary
| Phase | Issues | Est. Hours | Focus |
| --------- | -------------------- | ---------- | --------------- |
| 1 | KNOW-001 to KNOW-006 | 31h | CRUD + Basic UI |
| 2 | KNOW-007 to KNOW-012 | 24h | Wiki-links |
| 3 | KNOW-013 to KNOW-018 | 28h | Search |
| 4 | KNOW-019 to KNOW-022 | 19h | Graph |
| 5 | KNOW-023 to KNOW-028 | 25h | Polish |
| **Total** | 28 issues | ~127h | ~3-4 dev weeks |
---
_Generated by Jarvis • 2025-01-29_
Reference in New Issue View Git Blame Copy Permalink