Merge branch 'develop' into feature/semantic-search

2026-01-30 21:20:22 +00:00
parent 3ec2059470 26a7175744
commit eca6a9efe2
4 changed files with 3533 additions and 0 deletions
--- a/KNOWLEDGE_API.md
+++ b/KNOWLEDGE_API.md
--- a/KNOWLEDGE_DEV.md
+++ b/KNOWLEDGE_DEV.md
--- a/KNOWLEDGE_USER_GUIDE.md
+++ b/KNOWLEDGE_USER_GUIDE.md
@@ -0,0 +1,628 @@
+# Knowledge Module - User Guide
+
+The Knowledge Module is a powerful, personal wiki and knowledge management system built into Mosaic Stack. Create interconnected notes, organize with tags, track changes over time, and visualize relationships between your knowledge entries.
+
+## Table of Contents
+
+1. [Getting Started](#getting-started)
+2. [Creating Entries](#creating-entries)
+3. [Wiki-links and Backlinks](#wiki-links-and-backlinks)
+4. [Tags and Organization](#tags-and-organization)
+5. [Search](#search)
+6. [Import/Export](#importexport)
+7. [Version History](#version-history)
+8. [Graph Visualization](#graph-visualization)
+
+---
+
+## Getting Started
+
+The Knowledge Module provides a flexible way to capture and organize information:
+
+- **Markdown-based**: Write entries using Markdown for rich formatting
+- **Wiki-style linking**: Connect entries using `[[wiki-links]]`
+- **Tag-based organization**: Categorize entries with tags
+- **Full version history**: Every edit is tracked and recoverable
+- **Powerful search**: Find entries with full-text search
+- **Visual knowledge graph**: See relationships between entries
+- **Import/Export**: Bulk import/export for portability
+
+### Entry Lifecycle
+
+Each knowledge entry has three possible statuses:
+
+- **DRAFT** — Entry is being worked on, visible only to you
+- **PUBLISHED** — Entry is complete and visible to workspace members
+- **ARCHIVED** — Entry is hidden from normal views but preserved
+
+And three visibility levels:
+
+- **PRIVATE** — Only visible to you
+- **WORKSPACE** — Visible to all workspace members
+- **PUBLIC** — Visible to anyone (future feature)
+
+---
+
+## Creating Entries
+
+### Basic Entry Creation
+
+Every entry has:
+
+- **Title** (required) — The entry name (up to 500 characters)
+- **Content** (required) — Markdown-formatted text
+- **Summary** (optional) — Brief description (up to 1000 characters)
+- **Tags** (optional) — Categories for organization
+- **Status** — DRAFT, PUBLISHED, or ARCHIVED
+- **Visibility** — PRIVATE, WORKSPACE, or PUBLIC
+
+### Slugs
+
+When you create an entry, the system automatically generates a unique **slug** from the title:
+
+- `"My First Entry"` → `my-first-entry`
+- `"API Design Patterns"` → `api-design-patterns`
+- `"React Hooks Guide"` → `react-hooks-guide`
+
+Slugs are used in URLs and wiki-links. They're unique per workspace.
+
+### Example Entry
+
+```markdown
+Title: React Component Patterns
+
+Content:
+## Component Composition
+
+React components can be composed using several patterns:
+
+### Container/Presentational Pattern
+Separate data logic (containers) from UI (presentational).
+
+See also: [[React Hooks]], [[State Management]]
+
+Tags: react, frontend, patterns
+```
+
+### Change Notes
+
+When creating or updating entries, you can add an optional **change note** to describe what you changed:
+
+```
+"Added section on custom hooks"
+"Fixed typo in code example"
+"Initial draft"
+```
+
+Change notes appear in version history, making it easy to track why changes were made.
+
+---
+
+## Wiki-links and Backlinks
+
+### Creating Links
+
+Link to other entries using **wiki-link syntax**:
+
+```markdown
+[[Entry Title]]
+[[entry-slug]]
+[[Entry Title|Custom Link Text]]
+```
+
+Examples:
+
+```markdown
+For more details, see [[API Documentation]].
+Learn about [[react-hooks|React Hooks]].
+Related: [[Frontend Best Practices]], [[TypeScript Guide]]
+```
+
+### How Wiki-links Work
+
+1. **Automatic resolution**: The system finds the target entry by slug or title
+2. **Smart matching**: Links work with slugs (`react-hooks`) or titles (`React Hooks`)
+3. **Custom text**: Use `[[slug|display text]]` for custom link text
+4. **Auto-linking**: Links are parsed and resolved when you save the entry
+
+### Unresolved Links
+
+If you link to an entry that doesn't exist yet, it's marked as **unresolved**:
+
+```markdown
+[[Future Entry That Doesn't Exist Yet]]
+```
+
+Unresolved links:
+- Are still stored and tracked
+- Will automatically resolve when the target entry is created
+- Show as unlinked in the UI (implementation-specific)
+
+This lets you create links before creating the entries they point to!
+
+### Backlinks
+
+Every entry automatically tracks its **backlinks** — entries that link *to* it.
+
+**Example**: If entry "React Hooks" is linked from:
+- "Frontend Guide"
+- "Component Patterns"  
+- "State Management"
+
+Then "React Hooks" will show 3 backlinks.
+
+**Use backlinks to:**
+- Discover related content
+- Understand entry relationships
+- Navigate bidirectionally through knowledge
+
+Access backlinks via: `GET /api/knowledge/entries/:slug/backlinks`
+
+---
+
+## Tags and Organization
+
+### Creating Tags
+
+Tags help categorize and organize entries. Create tags with:
+
+- **Name** (required) — Display name (e.g., "Frontend Development")
+- **Slug** (auto-generated) — URL-friendly identifier (e.g., "frontend-development")
+- **Color** (optional) — Hex color for visual organization (e.g., "#3b82f6")
+- **Description** (optional) — Tag purpose or usage notes
+
+### Tagging Entries
+
+Add tags when creating or updating entries:
+
+```json
+{
+  "title": "React Component Guide",
+  "content": "...",
+  "tags": ["react", "frontend", "tutorial"]
+}
+```
+
+### Finding Tagged Entries
+
+Search for entries with specific tags:
+
+```
+GET /api/knowledge/search/by-tags?tags=react,frontend
+```
+
+This finds entries that have **ALL** specified tags (AND logic).
+
+### Tag Management
+
+- **List all tags**: `GET /api/knowledge/tags`
+- **Get tag details**: `GET /api/knowledge/tags/:slug`
+- **Get tagged entries**: `GET /api/knowledge/tags/:slug/entries`
+- **Update tag**: `PUT /api/knowledge/tags/:slug`
+- **Delete tag**: `DELETE /api/knowledge/tags/:slug` (admin only)
+
+Deleting a tag removes it from all entries but doesn't delete the entries themselves.
+
+---
+
+## Search
+
+The Knowledge Module provides powerful search capabilities:
+
+### Full-Text Search
+
+Search across entry titles and content with relevance ranking:
+
+```
+GET /api/knowledge/search?q=react hooks&page=1&limit=20
+```
+
+**Features:**
+- Searches **title** and **content** fields
+- Relevance ranking (best matches first)
+- Case-insensitive
+- Partial word matching
+- Pagination support
+
+**Query parameters:**
+- `q` (required) — Search query string
+- `status` (optional) — Filter by status (DRAFT, PUBLISHED, ARCHIVED)
+- `page` (default: 1) — Page number
+- `limit` (default: 20, max: 100) — Results per page
+
+### Tag-Based Search
+
+Find entries with specific tags:
+
+```
+GET /api/knowledge/search/by-tags?tags=react,typescript
+```
+
+Returns entries that have **ALL** specified tags.
+
+### Recent Entries
+
+Get recently modified entries:
+
+```
+GET /api/knowledge/search/recent?limit=10&status=PUBLISHED
+```
+
+**Parameters:**
+- `limit` (default: 10, max: 50) — Number of entries
+- `status` (optional) — Filter by status
+
+Perfect for "what's new" or "recently updated" views.
+
+### Combining Filters
+
+All search endpoints support status filtering:
+
+- `status=DRAFT` — Only draft entries
+- `status=PUBLISHED` — Only published entries
+- `status=ARCHIVED` — Only archived entries
+- (no status) — All statuses
+
+---
+
+## Import/Export
+
+### Exporting Entries
+
+Export your knowledge base for backup or migration:
+
+```
+GET /api/knowledge/export?format=markdown
+```
+
+**Export formats:**
+
+1. **Markdown** (default)
+   - Each entry saved as `.md` file
+   - Filename: `{slug}.md`
+   - Front matter with metadata (title, tags, status, etc.)
+   - Returns `.zip` archive
+
+2. **JSON**
+   - Structured JSON format
+   - Complete entry data including metadata
+   - Returns `.zip` archive
+
+**Export specific entries:**
+
+```
+GET /api/knowledge/export?format=markdown&entryIds[]=uuid1&entryIds[]=uuid2
+```
+
+If `entryIds` is omitted, exports **all entries** in the workspace.
+
+**Example Markdown export:**
+
+```markdown
+---
+slug: react-hooks-guide
+title: React Hooks Guide
+status: PUBLISHED
+visibility: WORKSPACE
+tags: react, frontend
+createdAt: 2024-01-29T10:00:00Z
+updatedAt: 2024-01-30T15:30:00Z
+---
+
+# React Hooks Guide
+
+Content goes here...
+
+[[Related Entry]]
+```
+
+### Importing Entries
+
+Import entries from `.md` or `.zip` files:
+
+```bash
+curl -X POST http://localhost:3001/api/knowledge/import \
+  -H "Authorization: Bearer YOUR_TOKEN" \
+  -H "x-workspace-id: WORKSPACE_ID" \
+  -F "file=@knowledge-export.zip"
+```
+
+**Supported formats:**
+
+1. **Single Markdown file** (`.md`)
+   - Creates one entry
+   - Reads front matter for metadata
+   - Generates slug from filename if not in front matter
+
+2. **Zip archive** (`.zip`)
+   - Multiple `.md` files
+   - Each file becomes one entry
+   - Front matter optional
+
+**Import behavior:**
+
+- **New entries**: Creates with status DRAFT
+- **Existing entries** (matching slug): Skipped (does not overwrite)
+- **Wiki-links**: Preserved and will resolve if targets exist
+- **Tags**: Created if they don't exist
+- **Validation**: Invalid entries are skipped with error details
+
+**Response:**
+
+```json
+{
+  "success": true,
+  "totalFiles": 10,
+  "imported": 8,
+  "failed": 2,
+  "results": [
+    {
+      "filename": "react-hooks.md",
+      "success": true,
+      "entryId": "uuid",
+      "slug": "react-hooks"
+    },
+    {
+      "filename": "invalid-entry.md",
+      "success": false,
+      "error": "Title is required"
+    }
+  ]
+}
+```
+
+### File Size Limits
+
+- Maximum file size: **50MB**
+- Accepted file types: `.md`, `.zip`
+
+---
+
+## Version History
+
+Every edit to a knowledge entry is automatically saved as a **version**. You can view history, compare changes, and restore previous versions.
+
+### How Versioning Works
+
+- **Automatic versioning**: Every update creates a new version
+- **Version numbers**: Auto-incremented (1, 2, 3, ...)
+- **What's tracked**: Title, content, summary
+- **Change notes**: Optional message describing the change
+- **Author tracking**: Who made each change
+- **Timestamps**: When each version was created
+
+### Viewing Version History
+
+**List all versions for an entry:**
+
+```
+GET /api/knowledge/entries/:slug/versions?page=1&limit=20
+```
+
+Returns paginated list of versions with:
+- Version number
+- Title
+- Summary
+- Change note
+- Author info
+- Timestamp
+
+**Get a specific version:**
+
+```
+GET /api/knowledge/entries/:slug/versions/:version
+```
+
+Returns the complete entry as it existed at that version:
+- Title
+- Content  
+- Summary
+- Change note
+- Author
+- Created timestamp
+
+### Restoring a Previous Version
+
+Restore an entry to a previous version:
+
+```
+POST /api/knowledge/entries/:slug/restore/:version
+Body: { "changeNote": "Restored version 5" }
+```
+
+**What happens:**
+1. Creates a **new version** with content from the specified version
+2. The change note is required to document why you restored
+3. Original versions remain intact (no data loss)
+4. Version numbers continue incrementing (no rewriting history)
+
+**Example:**
+- Current version: 10
+- Restore version 5
+- New version created: 11 (with content from version 5)
+
+### Best Practices
+
+- **Write meaningful change notes**: "Added examples" is better than "Updated"
+- **Review before publishing**: Keep entries in DRAFT while iterating
+- **Restore carefully**: Preview the old version before restoring
+- **Use versions for comparison**: See how entries evolved over time
+
+---
+
+## Graph Visualization
+
+The Knowledge Module includes a powerful **graph visualization** feature (currently available via service layer, REST endpoint coming soon).
+
+### How the Graph Works
+
+The knowledge graph represents:
+
+- **Nodes**: Knowledge entries
+- **Edges**: Wiki-links between entries
+- **Relationships**: Bidirectional (incoming and outgoing links)
+- **Depth traversal**: Explore connections up to N levels deep
+
+### Entry-Centered Graph
+
+Get a graph view centered on a specific entry:
+
+```typescript
+// Service layer (REST endpoint coming soon)
+const graph = await graphService.getEntryGraph(
+  workspaceId,
+  entryId,
+  maxDepth // default: 1
+);
+```
+
+**Response structure:**
+
+```typescript
+{
+  centerNode: {
+    id: "uuid",
+    slug: "react-hooks",
+    title: "React Hooks Guide",
+    summary: "Comprehensive guide to React Hooks",
+    tags: [
+      { id: "uuid", name: "React", slug: "react", color: "#61dafb" }
+    ],
+    depth: 0
+  },
+  nodes: [
+    // All connected entries up to maxDepth
+    { id: "uuid", slug: "...", title: "...", depth: 1 },
+    { id: "uuid", slug: "...", title: "...", depth: 2 }
+  ],
+  edges: [
+    {
+      id: "uuid",
+      sourceId: "entry1-uuid",
+      targetId: "entry2-uuid",
+      linkText: "React Hooks"
+    }
+  ],
+  stats: {
+    totalNodes: 15,
+    totalEdges: 22,
+    maxDepth: 2
+  }
+}
+```
+
+### Graph Properties
+
+- **Depth 0**: Just the center node (no connections)
+- **Depth 1**: Center node + directly connected entries
+- **Depth 2**: Depth 1 + entries connected to depth 1 nodes
+- **Depth N**: Continue expanding N levels
+
+**Node information:**
+- Entry metadata (slug, title, summary)
+- Tags with colors
+- Depth level from center
+
+**Edge information:**
+- Source and target entry IDs
+- Original link text from the markdown
+- Unique link identifier
+
+### Use Cases
+
+- **Discover connections**: Find related entries
+- **Visualize knowledge structure**: See how concepts relate
+- **Navigate bidirectionally**: Follow links in both directions
+- **Cluster analysis**: Identify knowledge hubs (highly connected entries)
+- **Content gap analysis**: Find isolated entries needing more connections
+
+### Performance & Caching
+
+Graph queries are **cached** for performance:
+
+- **Cache key**: `workspace:entry:depth`
+- **TTL**: 5 minutes (configurable)
+- **Invalidation**: Automatic on entry or link updates
+
+Large graphs (depth > 2) can be expensive. The cache ensures fast repeat access.
+
+---
+
+## Tips & Best Practices
+
+### Content Organization
+
+1. **Start with outlines**: Create stub entries, fill in later
+2. **Link early and often**: Wiki-links are cheap, use them liberally
+3. **Tag consistently**: Establish a tag taxonomy early
+4. **Write summaries**: Help future-you find content faster
+5. **Use DRAFT status**: Iterate privately before publishing
+
+### Naming Conventions
+
+- **Titles**: Clear, descriptive, unique
+- **Slugs**: Auto-generated, don't worry about them
+- **Tags**: Short, lowercase, consistent naming (e.g., `react` not `React` or `ReactJS`)
+
+### Knowledge Graph Health
+
+- **Avoid orphans**: Link new entries to existing content
+- **Create hubs**: Some entries naturally become central (index pages)
+- **Bidirectional linking**: Link both ways when relationships are mutual
+- **Tag hubs**: Use tags for broad categories, links for specific relationships
+
+### Workflow Patterns
+
+**Personal Wiki:**
+```
+Draft → Link → Tag → Publish → Iterate
+```
+
+**Team Knowledge Base:**
+```
+Draft → Review → Link → Tag → Publish → Maintain
+```
+
+**Research Notes:**
+```
+Capture → Organize → Synthesize → Link → Archive
+```
+
+---
+
+## Permissions
+
+Knowledge Module endpoints require specific permissions:
+
+- **Read** (ANY workspace member)
+  - List entries
+  - View entries
+  - View backlinks
+  - View versions
+  - Search
+  - Export
+
+- **Write** (MEMBER role or higher)
+  - Create entries
+  - Update entries
+  - Import entries
+  - Restore versions
+
+- **Delete/Admin** (ADMIN role or higher)
+  - Archive entries
+  - Delete entries
+  - Clear cache
+
+See [API Documentation](KNOWLEDGE_API.md) for complete endpoint permissions.
+
+---
+
+## Next Steps
+
+- **[API Documentation](KNOWLEDGE_API.md)** — Complete REST API reference
+- **[Developer Guide](KNOWLEDGE_DEV.md)** — Architecture and implementation details
+- **[Main README](README.md)** — Full Mosaic Stack documentation
+
+---
+
+**Happy knowledge building! 🧠✨**
--- a/README.md
+++ b/README.md
@@ -7,6 +7,7 @@ Multi-tenant personal assistant platform with PostgreSQL backend, Authentik SSO,
 Mosaic Stack is a modern, PDA-friendly platform designed to help users manage their personal and professional lives with:

 - **Multi-user workspaces** with team collaboration
+- **Knowledge management** with wiki-style linking and version history
 - **Task management** with flexible organization
 - **Event & calendar** integration
 - **Project tracking** with Gantt charts and Kanban boards
@@ -185,6 +186,111 @@ mosaic-stack/

 See the [issue tracker](https://git.mosaicstack.dev/mosaic/stack/issues) for complete roadmap.

+## Knowledge Module
+
+The **Knowledge Module** is a powerful personal wiki and knowledge management system built into Mosaic Stack. Create interconnected notes, organize with tags, track changes over time, and visualize relationships.
+
+### Features
+
+- **📝 Markdown-based entries** — Write using familiar Markdown syntax
+- **🔗 Wiki-style linking** — Connect entries using `[[wiki-links]]`
+- **🏷️ Tag organization** — Categorize and filter with flexible tagging
+- **📜 Full version history** — Every edit is tracked and recoverable
+- **🔍 Powerful search** — Full-text search across titles and content
+- **📊 Knowledge graph** — Visualize relationships between entries
+- **📤 Import/Export** — Bulk import/export for portability
+- **⚡ Valkey caching** — High-performance caching for fast access
+
+### Quick Examples
+
+**Create an entry:**
+```bash
+curl -X POST http://localhost:3001/api/knowledge/entries \
+  -H "Authorization: Bearer YOUR_TOKEN" \
+  -H "x-workspace-id: WORKSPACE_ID" \
+  -d '{
+    "title": "React Hooks Guide",
+    "content": "# React Hooks\n\nSee [[Component Patterns]] for more.",
+    "tags": ["react", "frontend"],
+    "status": "PUBLISHED"
+  }'
+```
+
+**Search entries:**
+```bash
+curl -X GET 'http://localhost:3001/api/knowledge/search?q=react+hooks' \
+  -H "Authorization: Bearer YOUR_TOKEN" \
+  -H "x-workspace-id: WORKSPACE_ID"
+```
+
+**Export knowledge base:**
+```bash
+curl -X GET 'http://localhost:3001/api/knowledge/export?format=markdown' \
+  -H "Authorization: Bearer YOUR_TOKEN" \
+  -H "x-workspace-id: WORKSPACE_ID" \
+  -o knowledge-export.zip
+```
+
+### Documentation
+
+- **[User Guide](KNOWLEDGE_USER_GUIDE.md)** — Getting started, features, and workflows
+- **[API Documentation](KNOWLEDGE_API.md)** — Complete REST API reference with examples
+- **[Developer Guide](KNOWLEDGE_DEV.md)** — Architecture, implementation, and contributing
+
+### Key Concepts
+
+**Wiki-links**
+Connect entries using double-bracket syntax:
+```markdown
+See [[Entry Title]] or [[entry-slug]] for details.
+Use [[Page|custom text]] for custom display text.
+```
+
+**Version History**
+Every edit creates a new version. View history, compare changes, and restore previous versions:
+```bash
+# List versions
+GET /api/knowledge/entries/:slug/versions
+
+# Get specific version
+GET /api/knowledge/entries/:slug/versions/:version
+
+# Restore version
+POST /api/knowledge/entries/:slug/restore/:version
+```
+
+**Backlinks**
+Automatically discover entries that link to a given entry:
+```bash
+GET /api/knowledge/entries/:slug/backlinks
+```
+
+**Tags**
+Organize entries with tags:
+```bash
+# Create tag
+POST /api/knowledge/tags
+{ "name": "React", "color": "#61dafb" }
+
+# Find entries with tags
+GET /api/knowledge/search/by-tags?tags=react,frontend
+```
+
+### Performance
+
+With Valkey caching enabled:
+- **Entry retrieval:** ~2-5ms (vs ~50ms uncached)
+- **Search queries:** ~2-5ms (vs ~200ms uncached)
+- **Graph traversals:** ~2-5ms (vs ~400ms uncached)
+- **Cache hit rates:** 70-90% for active workspaces
+
+Configure caching via environment variables:
+```bash
+VALKEY_URL=redis://localhost:6379
+KNOWLEDGE_CACHE_ENABLED=true
+KNOWLEDGE_CACHE_TTL=300  # 5 minutes
+```
+
 ## Development Workflow

 ### Branch Strategy