All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
1270 lines
32 KiB
Markdown
1270 lines
32 KiB
Markdown
# Knowledge Module - Developer Guide
|
|
|
|
Comprehensive developer documentation for the Knowledge Module implementation, architecture, and contribution guidelines.
|
|
|
|
## Table of Contents
|
|
|
|
1. [Architecture Overview](#architecture-overview)
|
|
2. [Database Schema](#database-schema)
|
|
3. [Service Layer](#service-layer)
|
|
4. [Caching Strategy](#caching-strategy)
|
|
5. [Wiki-Link System](#wiki-link-system)
|
|
6. [Testing Guide](#testing-guide)
|
|
7. [Contributing](#contributing)
|
|
|
|
---
|
|
|
|
## Architecture Overview
|
|
|
|
The Knowledge Module follows a **layered architecture** pattern:
|
|
|
|
```
|
|
┌─────────────────────────────────────────┐
|
|
│ Controllers (REST) │
|
|
│ knowledge | search | tags | import │
|
|
└────────────────┬────────────────────────┘
|
|
│
|
|
┌────────────────▼────────────────────────┐
|
|
│ Service Layer │
|
|
│ KnowledgeService | SearchService │
|
|
│ LinkSyncService | GraphService │
|
|
│ TagsService | ImportExportService │
|
|
│ StatsService | CacheService │
|
|
└────────────────┬────────────────────────┘
|
|
│
|
|
┌────────────────▼────────────────────────┐
|
|
│ Data Access (Prisma ORM) │
|
|
│ KnowledgeEntry | KnowledgeLink │
|
|
│ KnowledgeTag | KnowledgeEntryVersion │
|
|
│ KnowledgeEmbedding │
|
|
└────────────────┬────────────────────────┘
|
|
│
|
|
┌────────────────▼────────────────────────┐
|
|
│ PostgreSQL 17 + pgvector │
|
|
└─────────────────────────────────────────┘
|
|
```
|
|
|
|
### Module Structure
|
|
|
|
```
|
|
apps/api/src/knowledge/
|
|
├── controllers/
|
|
│ ├── knowledge.controller.ts # Entry CRUD endpoints
|
|
│ ├── search.controller.ts # Search endpoints
|
|
│ ├── tags.controller.ts # Tag management
|
|
│ ├── import-export.controller.ts # Import/export
|
|
│ └── stats.controller.ts # Statistics
|
|
├── services/
|
|
│ ├── cache.service.ts # Valkey caching
|
|
│ ├── graph.service.ts # Graph traversal
|
|
│ ├── import-export.service.ts # File import/export
|
|
│ ├── link-resolution.service.ts # Link resolution
|
|
│ ├── link-sync.service.ts # Link synchronization
|
|
│ ├── search.service.ts # Full-text search
|
|
│ └── stats.service.ts # Statistics aggregation
|
|
├── entities/
|
|
│ ├── knowledge-entry.entity.ts # Entry DTOs
|
|
│ ├── knowledge-entry-version.entity.ts
|
|
│ ├── graph.entity.ts # Graph DTOs
|
|
│ └── stats.entity.ts # Stats DTOs
|
|
├── dto/
|
|
│ ├── create-entry.dto.ts
|
|
│ ├── update-entry.dto.ts
|
|
│ ├── entry-query.dto.ts
|
|
│ ├── search-query.dto.ts
|
|
│ └── ...
|
|
├── utils/
|
|
│ ├── wiki-link-parser.ts # Wiki-link parsing
|
|
│ └── markdown.ts # Markdown rendering
|
|
├── knowledge.service.ts # Core entry service
|
|
├── tags.service.ts # Tag service
|
|
└── knowledge.module.ts # NestJS module
|
|
```
|
|
|
|
### Key Responsibilities
|
|
|
|
**Controllers**
|
|
|
|
- HTTP request/response handling
|
|
- Input validation (DTOs)
|
|
- Permission enforcement (guards)
|
|
- Error handling
|
|
|
|
**Services**
|
|
|
|
- Business logic
|
|
- Data transformation
|
|
- Transaction management
|
|
- Cache invalidation
|
|
|
|
**Repositories (Prisma)**
|
|
|
|
- Database queries
|
|
- Relation loading
|
|
- Type-safe data access
|
|
|
|
---
|
|
|
|
## Database Schema
|
|
|
|
### Core Models
|
|
|
|
#### KnowledgeEntry
|
|
|
|
Main entity for knowledge base entries.
|
|
|
|
```prisma
|
|
model KnowledgeEntry {
|
|
id String @id @default(uuid()) @db.Uuid
|
|
workspaceId String @map("workspace_id") @db.Uuid
|
|
workspace Workspace @relation(fields: [workspaceId], references: [id], onDelete: Cascade)
|
|
|
|
// Identity
|
|
slug String // URL-friendly identifier
|
|
title String // Display name
|
|
|
|
// Content
|
|
content String @db.Text // Raw markdown
|
|
contentHtml String? @map("content_html") @db.Text // Rendered HTML
|
|
summary String? // Optional brief description
|
|
|
|
// Status
|
|
status EntryStatus @default(DRAFT) // DRAFT | PUBLISHED | ARCHIVED
|
|
visibility Visibility @default(PRIVATE) // PRIVATE | WORKSPACE | PUBLIC
|
|
|
|
// Audit
|
|
createdAt DateTime @default(now()) @map("created_at") @db.Timestamptz
|
|
updatedAt DateTime @updatedAt @map("updated_at") @db.Timestamptz
|
|
createdBy String @map("created_by") @db.Uuid
|
|
updatedBy String @map("updated_by") @db.Uuid
|
|
|
|
// Relations
|
|
tags KnowledgeEntryTag[]
|
|
outgoingLinks KnowledgeLink[] @relation("SourceEntry")
|
|
incomingLinks KnowledgeLink[] @relation("TargetEntry")
|
|
versions KnowledgeEntryVersion[]
|
|
embedding KnowledgeEmbedding?
|
|
|
|
@@unique([workspaceId, slug])
|
|
@@index([workspaceId, status])
|
|
@@index([workspaceId, updatedAt])
|
|
@@map("knowledge_entries")
|
|
}
|
|
```
|
|
|
|
**Indexes:**
|
|
|
|
- `workspaceId, slug` (unique constraint)
|
|
- `workspaceId, status` (filtering)
|
|
- `workspaceId, updatedAt` (recent entries)
|
|
|
|
#### KnowledgeLink
|
|
|
|
Represents wiki-links between entries.
|
|
|
|
```prisma
|
|
model KnowledgeLink {
|
|
id String @id @default(uuid()) @db.Uuid
|
|
|
|
sourceId String @map("source_id") @db.Uuid
|
|
source KnowledgeEntry @relation("SourceEntry", fields: [sourceId], references: [id], onDelete: Cascade)
|
|
|
|
targetId String @map("target_id") @db.Uuid
|
|
target KnowledgeEntry @relation("TargetEntry", fields: [targetId], references: [id], onDelete: Cascade)
|
|
|
|
// Link metadata
|
|
linkText String @map("link_text") // Original link text from markdown
|
|
context String? // Surrounding text (future feature)
|
|
|
|
createdAt DateTime @default(now()) @map("created_at") @db.Timestamptz
|
|
|
|
@@unique([sourceId, targetId])
|
|
@@index([sourceId])
|
|
@@index([targetId])
|
|
@@map("knowledge_links")
|
|
}
|
|
```
|
|
|
|
**Unique constraint:**
|
|
|
|
- `sourceId, targetId` (prevents duplicate links)
|
|
|
|
**Indexes:**
|
|
|
|
- `sourceId` (outgoing links lookup)
|
|
- `targetId` (backlinks lookup)
|
|
|
|
#### KnowledgeEntryVersion
|
|
|
|
Version history for entries.
|
|
|
|
```prisma
|
|
model KnowledgeEntryVersion {
|
|
id String @id @default(uuid()) @db.Uuid
|
|
entryId String @map("entry_id") @db.Uuid
|
|
entry KnowledgeEntry @relation(fields: [entryId], references: [id], onDelete: Cascade)
|
|
|
|
version Int // Version number (1, 2, 3, ...)
|
|
title String
|
|
content String @db.Text
|
|
summary String?
|
|
|
|
createdAt DateTime @default(now()) @map("created_at") @db.Timestamptz
|
|
createdBy String @map("created_by") @db.Uuid
|
|
author User @relation("EntryVersionAuthor", fields: [createdBy], references: [id])
|
|
changeNote String? @map("change_note") // Optional change description
|
|
|
|
@@unique([entryId, version])
|
|
@@index([entryId, version])
|
|
@@map("knowledge_entry_versions")
|
|
}
|
|
```
|
|
|
|
**Versioning strategy:**
|
|
|
|
- Auto-incrementing version numbers
|
|
- Immutable history (no updates or deletes)
|
|
- Snapshot of title, content, summary at time of save
|
|
|
|
#### KnowledgeTag
|
|
|
|
Tags for categorization.
|
|
|
|
```prisma
|
|
model KnowledgeTag {
|
|
id String @id @default(uuid()) @db.Uuid
|
|
workspaceId String @map("workspace_id") @db.Uuid
|
|
workspace Workspace @relation(fields: [workspaceId], references: [id], onDelete: Cascade)
|
|
|
|
name String // Display name
|
|
slug String // URL-friendly identifier
|
|
color String? // Hex color (e.g., "#3b82f6")
|
|
description String?
|
|
|
|
entries KnowledgeEntryTag[]
|
|
|
|
@@unique([workspaceId, slug])
|
|
@@index([workspaceId])
|
|
@@map("knowledge_tags")
|
|
}
|
|
```
|
|
|
|
#### KnowledgeEntryTag
|
|
|
|
Many-to-many junction table for entries and tags.
|
|
|
|
```prisma
|
|
model KnowledgeEntryTag {
|
|
entryId String @map("entry_id") @db.Uuid
|
|
entry KnowledgeEntry @relation(fields: [entryId], references: [id], onDelete: Cascade)
|
|
|
|
tagId String @map("tag_id") @db.Uuid
|
|
tag KnowledgeTag @relation(fields: [tagId], references: [id], onDelete: Cascade)
|
|
|
|
@@id([entryId, tagId])
|
|
@@index([entryId])
|
|
@@index([tagId])
|
|
@@map("knowledge_entry_tags")
|
|
}
|
|
```
|
|
|
|
#### KnowledgeEmbedding
|
|
|
|
Semantic search embeddings (future feature).
|
|
|
|
```prisma
|
|
model KnowledgeEmbedding {
|
|
id String @id @default(uuid()) @db.Uuid
|
|
entryId String @unique @map("entry_id") @db.Uuid
|
|
entry KnowledgeEntry @relation(fields: [entryId], references: [id], onDelete: Cascade)
|
|
|
|
embedding Unsupported("vector(1536)") // pgvector type
|
|
model String // Model used (e.g., "text-embedding-ada-002")
|
|
|
|
createdAt DateTime @default(now()) @map("created_at") @db.Timestamptz
|
|
updatedAt DateTime @updatedAt @map("updated_at") @db.Timestamptz
|
|
|
|
@@index([entryId])
|
|
@@map("knowledge_embeddings")
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Service Layer
|
|
|
|
### KnowledgeService
|
|
|
|
Core service for entry management.
|
|
|
|
**Key methods:**
|
|
|
|
```typescript
|
|
class KnowledgeService {
|
|
// CRUD operations
|
|
async findAll(workspaceId, query): Promise<PaginatedEntries>;
|
|
async findOne(workspaceId, slug): Promise<KnowledgeEntryWithTags>;
|
|
async create(workspaceId, userId, dto): Promise<KnowledgeEntryWithTags>;
|
|
async update(workspaceId, slug, userId, dto): Promise<KnowledgeEntryWithTags>;
|
|
async remove(workspaceId, slug, userId): Promise<void>;
|
|
|
|
// Version management
|
|
async findVersions(workspaceId, slug, page, limit): Promise<PaginatedVersions>;
|
|
async findVersion(workspaceId, slug, version): Promise<KnowledgeEntryVersion>;
|
|
async restoreVersion(
|
|
workspaceId,
|
|
slug,
|
|
version,
|
|
userId,
|
|
changeNote
|
|
): Promise<KnowledgeEntryWithTags>;
|
|
}
|
|
```
|
|
|
|
**Create flow:**
|
|
|
|
```typescript
|
|
async create(workspaceId, userId, dto) {
|
|
// 1. Generate slug from title
|
|
const slug = this.generateUniqueSlug(dto.title, workspaceId);
|
|
|
|
// 2. Render markdown to HTML
|
|
const contentHtml = renderMarkdown(dto.content);
|
|
|
|
// 3. Create entry in transaction
|
|
const entry = await this.prisma.$transaction(async (tx) => {
|
|
// Create entry
|
|
const newEntry = await tx.knowledgeEntry.create({
|
|
data: {
|
|
workspaceId,
|
|
slug,
|
|
title: dto.title,
|
|
content: dto.content,
|
|
contentHtml,
|
|
summary: dto.summary,
|
|
status: dto.status || 'DRAFT',
|
|
visibility: dto.visibility || 'PRIVATE',
|
|
createdBy: userId,
|
|
updatedBy: userId,
|
|
},
|
|
});
|
|
|
|
// Create initial version (v1)
|
|
await tx.knowledgeEntryVersion.create({
|
|
data: {
|
|
entryId: newEntry.id,
|
|
version: 1,
|
|
title: newEntry.title,
|
|
content: newEntry.content,
|
|
summary: newEntry.summary,
|
|
createdBy: userId,
|
|
changeNote: dto.changeNote || 'Initial version',
|
|
},
|
|
});
|
|
|
|
// Handle tags (create or link)
|
|
if (dto.tags && dto.tags.length > 0) {
|
|
await this.linkTags(tx, newEntry.id, workspaceId, dto.tags);
|
|
}
|
|
|
|
return newEntry;
|
|
});
|
|
|
|
// 4. Parse and sync wiki-links (outside transaction)
|
|
await this.linkSync.syncLinks(entry.id, dto.content);
|
|
|
|
// 5. Invalidate caches
|
|
await this.cache.invalidateSearchCaches(workspaceId);
|
|
|
|
// 6. Return with tags
|
|
return this.findOne(workspaceId, slug);
|
|
}
|
|
```
|
|
|
|
**Update flow:**
|
|
|
|
```typescript
|
|
async update(workspaceId, slug, userId, dto) {
|
|
// 1. Find existing entry
|
|
const existing = await this.findOne(workspaceId, slug);
|
|
|
|
// 2. Get next version number
|
|
const latestVersion = await this.getLatestVersion(existing.id);
|
|
const nextVersion = latestVersion.version + 1;
|
|
|
|
// 3. Render new HTML if content changed
|
|
const contentHtml = dto.content
|
|
? renderMarkdown(dto.content)
|
|
: existing.contentHtml;
|
|
|
|
// 4. Update in transaction
|
|
const updated = await this.prisma.$transaction(async (tx) => {
|
|
// Update entry
|
|
const updatedEntry = await tx.knowledgeEntry.update({
|
|
where: { id: existing.id },
|
|
data: {
|
|
title: dto.title ?? existing.title,
|
|
content: dto.content ?? existing.content,
|
|
contentHtml,
|
|
summary: dto.summary ?? existing.summary,
|
|
status: dto.status ?? existing.status,
|
|
visibility: dto.visibility ?? existing.visibility,
|
|
updatedBy: userId,
|
|
},
|
|
});
|
|
|
|
// Create version snapshot
|
|
await tx.knowledgeEntryVersion.create({
|
|
data: {
|
|
entryId: updatedEntry.id,
|
|
version: nextVersion,
|
|
title: updatedEntry.title,
|
|
content: updatedEntry.content,
|
|
summary: updatedEntry.summary,
|
|
createdBy: userId,
|
|
changeNote: dto.changeNote || `Update to version ${nextVersion}`,
|
|
},
|
|
});
|
|
|
|
// Update tags if provided
|
|
if (dto.tags !== undefined) {
|
|
await this.replaceTags(tx, updatedEntry.id, workspaceId, dto.tags);
|
|
}
|
|
|
|
return updatedEntry;
|
|
});
|
|
|
|
// 5. Re-sync links if content changed
|
|
if (dto.content) {
|
|
await this.linkSync.syncLinks(updated.id, dto.content);
|
|
}
|
|
|
|
// 6. Invalidate caches
|
|
await this.cache.invalidateEntry(workspaceId, slug);
|
|
await this.cache.invalidateSearchCaches(workspaceId);
|
|
await this.cache.invalidateGraphCachesForEntry(updated.id);
|
|
|
|
// 7. Return updated entry
|
|
return this.findOne(workspaceId, slug);
|
|
}
|
|
```
|
|
|
|
### LinkSyncService
|
|
|
|
Manages wiki-link parsing and synchronization.
|
|
|
|
**Key methods:**
|
|
|
|
```typescript
|
|
class LinkSyncService {
|
|
async syncLinks(entryId: string, content: string): Promise<void>;
|
|
async getBacklinks(entryId: string): Promise<BacklinkWithSource[]>;
|
|
}
|
|
```
|
|
|
|
**Link sync flow:**
|
|
|
|
```typescript
|
|
async syncLinks(entryId, content) {
|
|
// 1. Parse wiki-links from content
|
|
const parsedLinks = parseWikiLinks(content);
|
|
|
|
// 2. Get source entry details
|
|
const entry = await this.prisma.knowledgeEntry.findUnique({
|
|
where: { id: entryId },
|
|
select: { workspaceId: true },
|
|
});
|
|
|
|
// 3. Delete existing links from this entry
|
|
await this.prisma.knowledgeLink.deleteMany({
|
|
where: { sourceId: entryId },
|
|
});
|
|
|
|
// 4. For each parsed link:
|
|
for (const link of parsedLinks) {
|
|
// Try to resolve target entry
|
|
const target = await this.linkResolver.resolve(
|
|
link.target,
|
|
entry.workspaceId
|
|
);
|
|
|
|
if (target) {
|
|
// Create resolved link
|
|
await this.prisma.knowledgeLink.create({
|
|
data: {
|
|
sourceId: entryId,
|
|
targetId: target.id,
|
|
linkText: link.displayText,
|
|
},
|
|
});
|
|
}
|
|
// Note: Unresolved links are simply not created
|
|
// They may resolve later when target entry is created
|
|
}
|
|
|
|
// 5. Invalidate graph caches
|
|
await this.cache.invalidateGraphCachesForEntry(entryId);
|
|
}
|
|
```
|
|
|
|
### LinkResolutionService
|
|
|
|
Resolves wiki-link targets to actual entries.
|
|
|
|
**Resolution strategy:**
|
|
|
|
```typescript
|
|
async resolve(target: string, workspaceId: string) {
|
|
// Strategy 1: Match by exact slug
|
|
let entry = await this.prisma.knowledgeEntry.findUnique({
|
|
where: {
|
|
workspaceId_slug: { workspaceId, slug: target },
|
|
},
|
|
});
|
|
|
|
if (entry) return entry;
|
|
|
|
// Strategy 2: Generate slug from target and try again
|
|
const slugified = slugify(target, { lower: true, strict: true });
|
|
entry = await this.prisma.knowledgeEntry.findUnique({
|
|
where: {
|
|
workspaceId_slug: { workspaceId, slug: slugified },
|
|
},
|
|
});
|
|
|
|
if (entry) return entry;
|
|
|
|
// Strategy 3: Match by title (case-insensitive)
|
|
entry = await this.prisma.knowledgeEntry.findFirst({
|
|
where: {
|
|
workspaceId,
|
|
title: {
|
|
equals: target,
|
|
mode: 'insensitive',
|
|
},
|
|
},
|
|
});
|
|
|
|
return entry || null;
|
|
}
|
|
```
|
|
|
|
**Resolution examples:**
|
|
|
|
| Link Target | Resolution |
|
|
| ------------- | -------------------------------------------- |
|
|
| `react-hooks` | Exact slug match |
|
|
| `React Hooks` | Slugify to `react-hooks`, then match |
|
|
| `REACT HOOKS` | Case-insensitive title match → `React Hooks` |
|
|
|
|
### SearchService
|
|
|
|
Full-text search and filtering.
|
|
|
|
**Key methods:**
|
|
|
|
```typescript
|
|
class SearchService {
|
|
async search(query, workspaceId, options): Promise<PaginatedSearchResults>;
|
|
async searchByTags(tags, workspaceId, options): Promise<PaginatedEntries>;
|
|
async recentEntries(workspaceId, limit, status?): Promise<KnowledgeEntryWithTags[]>;
|
|
}
|
|
```
|
|
|
|
**Search implementation:**
|
|
|
|
```typescript
|
|
async search(query, workspaceId, options) {
|
|
// Check cache first
|
|
const cached = await this.cache.getSearch(workspaceId, query, options);
|
|
if (cached) return cached;
|
|
|
|
// Build where clause
|
|
const where: Prisma.KnowledgeEntryWhereInput = {
|
|
workspaceId,
|
|
OR: [
|
|
{ title: { contains: query, mode: 'insensitive' } },
|
|
{ content: { contains: query, mode: 'insensitive' } },
|
|
],
|
|
};
|
|
|
|
if (options.status) {
|
|
where.status = options.status;
|
|
}
|
|
|
|
// Execute search with pagination
|
|
const [entries, total] = await Promise.all([
|
|
this.prisma.knowledgeEntry.findMany({
|
|
where,
|
|
include: { tags: { include: { tag: true } } },
|
|
orderBy: { updatedAt: 'desc' },
|
|
skip: (options.page - 1) * options.limit,
|
|
take: options.limit,
|
|
}),
|
|
this.prisma.knowledgeEntry.count({ where }),
|
|
]);
|
|
|
|
const result = {
|
|
data: entries,
|
|
pagination: {
|
|
page: options.page,
|
|
limit: options.limit,
|
|
total,
|
|
totalPages: Math.ceil(total / options.limit),
|
|
},
|
|
};
|
|
|
|
// Cache the result
|
|
await this.cache.setSearch(workspaceId, query, options, result);
|
|
|
|
return result;
|
|
}
|
|
```
|
|
|
|
### GraphService
|
|
|
|
Knowledge graph traversal.
|
|
|
|
**Key methods:**
|
|
|
|
```typescript
|
|
class GraphService {
|
|
async getEntryGraph(
|
|
workspaceId: string,
|
|
entryId: string,
|
|
maxDepth: number = 1
|
|
): Promise<EntryGraphResponse>;
|
|
}
|
|
```
|
|
|
|
**BFS graph traversal:**
|
|
|
|
```typescript
|
|
async getEntryGraph(workspaceId, entryId, maxDepth) {
|
|
// Check cache
|
|
const cached = await this.cache.getGraph(workspaceId, entryId, maxDepth);
|
|
if (cached) return cached;
|
|
|
|
const nodes: GraphNode[] = [];
|
|
const edges: GraphEdge[] = [];
|
|
const visited = new Set<string>();
|
|
const queue: Array<[string, number]> = [[entryId, 0]];
|
|
|
|
visited.add(entryId);
|
|
|
|
while (queue.length > 0) {
|
|
const [currentId, depth] = queue.shift()!;
|
|
|
|
// Fetch entry with links
|
|
const entry = await this.prisma.knowledgeEntry.findUnique({
|
|
where: { id: currentId },
|
|
include: {
|
|
tags: { include: { tag: true } },
|
|
outgoingLinks: { include: { target: true } },
|
|
incomingLinks: { include: { source: true } },
|
|
},
|
|
});
|
|
|
|
if (!entry) continue;
|
|
|
|
// Add node
|
|
nodes.push({
|
|
id: entry.id,
|
|
slug: entry.slug,
|
|
title: entry.title,
|
|
summary: entry.summary,
|
|
tags: entry.tags.map(et => ({
|
|
id: et.tag.id,
|
|
name: et.tag.name,
|
|
slug: et.tag.slug,
|
|
color: et.tag.color,
|
|
})),
|
|
depth,
|
|
});
|
|
|
|
// Continue BFS if not at max depth
|
|
if (depth < maxDepth) {
|
|
// Process outgoing links
|
|
for (const link of entry.outgoingLinks) {
|
|
edges.push({
|
|
id: link.id,
|
|
sourceId: link.sourceId,
|
|
targetId: link.targetId,
|
|
linkText: link.linkText,
|
|
});
|
|
|
|
if (!visited.has(link.targetId)) {
|
|
visited.add(link.targetId);
|
|
queue.push([link.targetId, depth + 1]);
|
|
}
|
|
}
|
|
|
|
// Process incoming links
|
|
for (const link of entry.incomingLinks) {
|
|
const edgeExists = edges.some(
|
|
e => e.sourceId === link.sourceId && e.targetId === link.targetId
|
|
);
|
|
if (!edgeExists) {
|
|
edges.push({
|
|
id: link.id,
|
|
sourceId: link.sourceId,
|
|
targetId: link.targetId,
|
|
linkText: link.linkText,
|
|
});
|
|
}
|
|
|
|
if (!visited.has(link.sourceId)) {
|
|
visited.add(link.sourceId);
|
|
queue.push([link.sourceId, depth + 1]);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
const result = {
|
|
centerNode: nodes.find(n => n.id === entryId)!,
|
|
nodes,
|
|
edges,
|
|
stats: {
|
|
totalNodes: nodes.length,
|
|
totalEdges: edges.length,
|
|
maxDepth,
|
|
},
|
|
};
|
|
|
|
// Cache result
|
|
await this.cache.setGraph(workspaceId, entryId, maxDepth, result);
|
|
|
|
return result;
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Caching Strategy
|
|
|
|
The Knowledge Module uses **Valkey** (Redis-compatible) for high-performance caching.
|
|
|
|
### Cache Keys
|
|
|
|
```
|
|
knowledge:entry:{workspaceId}:{slug}
|
|
knowledge:search:{workspaceId}:{query}:{options_hash}
|
|
knowledge:search-tags:{workspaceId}:{tags}:{options_hash}
|
|
knowledge:graph:{workspaceId}:{entryId}:{depth}
|
|
```
|
|
|
|
### Cache Configuration
|
|
|
|
```typescript
|
|
class KnowledgeCacheService {
|
|
private readonly ENTRY_PREFIX = "knowledge:entry:";
|
|
private readonly SEARCH_PREFIX = "knowledge:search:";
|
|
private readonly SEARCH_TAGS_PREFIX = "knowledge:search-tags:";
|
|
private readonly GRAPH_PREFIX = "knowledge:graph:";
|
|
|
|
private readonly DEFAULT_TTL = 300; // 5 minutes
|
|
private readonly isEnabled: boolean;
|
|
|
|
constructor() {
|
|
this.isEnabled = process.env.KNOWLEDGE_CACHE_ENABLED !== "false";
|
|
this.DEFAULT_TTL = parseInt(process.env.KNOWLEDGE_CACHE_TTL || "300");
|
|
}
|
|
}
|
|
```
|
|
|
|
### Invalidation Strategy
|
|
|
|
**Entry changes:**
|
|
|
|
- **Create**: Invalidate search caches
|
|
- **Update**: Invalidate entry cache, search caches, graph caches
|
|
- **Delete**: Invalidate entry cache, search caches, graph caches
|
|
|
|
**Link changes:**
|
|
|
|
- Invalidate graph caches for source and target entries
|
|
|
|
**Tag changes:**
|
|
|
|
- Invalidate tag-based search caches
|
|
|
|
```typescript
|
|
async invalidateEntry(workspaceId: string, slug: string) {
|
|
const key = `${this.ENTRY_PREFIX}${workspaceId}:${slug}`;
|
|
await this.valkey.del(key);
|
|
this.stats.deletes++;
|
|
}
|
|
|
|
async invalidateSearchCaches(workspaceId: string) {
|
|
const pattern = `${this.SEARCH_PREFIX}${workspaceId}:*`;
|
|
const keys = await this.valkey.keys(pattern);
|
|
if (keys.length > 0) {
|
|
await this.valkey.del(...keys);
|
|
this.stats.deletes += keys.length;
|
|
}
|
|
}
|
|
|
|
async invalidateGraphCachesForEntry(entryId: string) {
|
|
// Graph caches include entryId in the key
|
|
const pattern = `${this.GRAPH_PREFIX}*:${entryId}:*`;
|
|
const keys = await this.valkey.keys(pattern);
|
|
if (keys.length > 0) {
|
|
await this.valkey.del(...keys);
|
|
this.stats.deletes += keys.length;
|
|
}
|
|
}
|
|
```
|
|
|
|
### Performance Metrics
|
|
|
|
Track cache effectiveness:
|
|
|
|
```typescript
|
|
interface CacheStats {
|
|
hits: number;
|
|
misses: number;
|
|
sets: number;
|
|
deletes: number;
|
|
hitRate: number;
|
|
}
|
|
|
|
getStats(): CacheStats {
|
|
const total = this.stats.hits + this.stats.misses;
|
|
return {
|
|
...this.stats,
|
|
hitRate: total > 0 ? this.stats.hits / total : 0,
|
|
};
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Wiki-Link System
|
|
|
|
### Parsing Algorithm
|
|
|
|
The wiki-link parser handles complex edge cases:
|
|
|
|
**Supported syntax:**
|
|
|
|
```markdown
|
|
[[Page Name]] → Link to "Page Name"
|
|
[[page-slug]] → Link by slug
|
|
[[Page Name|Display Text]] → Custom display text
|
|
```
|
|
|
|
**Edge cases handled:**
|
|
|
|
- Nested brackets: `[[Link with [brackets] inside]]`
|
|
- Code blocks: `` `[[not a link]]` ``
|
|
- Fenced code: ` ```[[not a link]]``` `
|
|
- Escaped brackets: `\[[not a link]]`
|
|
- Triple brackets: `[[[not a link]]]`
|
|
|
|
**Parsing flow:**
|
|
|
|
1. **Find excluded regions** (code blocks, inline code)
|
|
2. **Scan for `[[` patterns**
|
|
3. **Find matching `]]`**
|
|
4. **Validate link target**
|
|
5. **Parse pipe separator for display text**
|
|
6. **Return array of WikiLink objects**
|
|
|
|
```typescript
|
|
interface WikiLink {
|
|
raw: string; // "[[Page Name]]"
|
|
target: string; // "Page Name" or "page-slug"
|
|
displayText: string; // "Page Name" or custom text
|
|
start: number; // Position in content
|
|
end: number; // Position in content
|
|
}
|
|
```
|
|
|
|
### Link Resolution
|
|
|
|
**Three-step resolution:**
|
|
|
|
```
|
|
1. Exact slug match: "react-hooks" → entry with slug "react-hooks"
|
|
2. Slugified match: "React Hooks" → slugify → "react-hooks" → match
|
|
3. Title match: Case-insensitive title search
|
|
```
|
|
|
|
**Unresolved links:**
|
|
|
|
- Not stored in database
|
|
- Will auto-resolve when target entry is created
|
|
- Future: UI indication for broken links
|
|
|
|
### Link Synchronization
|
|
|
|
**On entry create/update:**
|
|
|
|
```
|
|
1. Parse wiki-links from content
|
|
2. Delete existing links from this entry
|
|
3. For each parsed link:
|
|
a. Try to resolve target
|
|
b. If resolved, create KnowledgeLink record
|
|
c. If unresolved, skip (may resolve later)
|
|
4. Invalidate graph caches
|
|
```
|
|
|
|
**Why delete-and-recreate?**
|
|
|
|
- Simpler than diffing changes
|
|
- Ensures consistency with current content
|
|
- Links are cheap to recreate
|
|
|
|
---
|
|
|
|
## Testing Guide
|
|
|
|
### Test Structure
|
|
|
|
```
|
|
apps/api/src/knowledge/
|
|
├── knowledge.service.spec.ts
|
|
├── tags.service.spec.ts
|
|
├── search.controller.spec.ts
|
|
├── tags.controller.spec.ts
|
|
├── services/
|
|
│ ├── cache.service.spec.ts
|
|
│ ├── graph.service.spec.ts
|
|
│ ├── link-resolution.service.spec.ts
|
|
│ ├── link-sync.service.spec.ts
|
|
│ └── search.service.spec.ts
|
|
└── utils/
|
|
├── markdown.spec.ts
|
|
└── wiki-link-parser.spec.ts
|
|
```
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
# All knowledge module tests
|
|
pnpm test knowledge
|
|
|
|
# Specific test file
|
|
pnpm test knowledge.service.spec.ts
|
|
|
|
# Watch mode
|
|
pnpm test:watch knowledge
|
|
|
|
# Coverage
|
|
pnpm test:coverage
|
|
```
|
|
|
|
### Test Coverage Requirements
|
|
|
|
- **Minimum:** 85% overall coverage
|
|
- **Critical paths:** 100% coverage required
|
|
- Entry CRUD operations
|
|
- Version management
|
|
- Link resolution
|
|
- Wiki-link parsing
|
|
|
|
### Writing Tests
|
|
|
|
**Service tests** (unit):
|
|
|
|
```typescript
|
|
describe("KnowledgeService", () => {
|
|
let service: KnowledgeService;
|
|
let prisma: PrismaService;
|
|
let linkSync: LinkSyncService;
|
|
let cache: KnowledgeCacheService;
|
|
|
|
beforeEach(async () => {
|
|
const module = await Test.createTestingModule({
|
|
providers: [
|
|
KnowledgeService,
|
|
{
|
|
provide: PrismaService,
|
|
useValue: mockDeep<PrismaService>(),
|
|
},
|
|
{
|
|
provide: LinkSyncService,
|
|
useValue: mockDeep<LinkSyncService>(),
|
|
},
|
|
{
|
|
provide: KnowledgeCacheService,
|
|
useValue: mockDeep<KnowledgeCacheService>(),
|
|
},
|
|
],
|
|
}).compile();
|
|
|
|
service = module.get(KnowledgeService);
|
|
prisma = module.get(PrismaService);
|
|
linkSync = module.get(LinkSyncService);
|
|
cache = module.get(KnowledgeCacheService);
|
|
});
|
|
|
|
describe("create", () => {
|
|
it("should create entry with unique slug", async () => {
|
|
const dto = {
|
|
title: "Test Entry",
|
|
content: "# Test",
|
|
};
|
|
|
|
prisma.knowledgeEntry.create.mockResolvedValue({
|
|
id: "entry-id",
|
|
slug: "test-entry",
|
|
...dto,
|
|
});
|
|
|
|
const result = await service.create("workspace-id", "user-id", dto);
|
|
|
|
expect(result.slug).toBe("test-entry");
|
|
expect(linkSync.syncLinks).toHaveBeenCalledWith("entry-id", dto.content);
|
|
expect(cache.invalidateSearchCaches).toHaveBeenCalled();
|
|
});
|
|
});
|
|
});
|
|
```
|
|
|
|
**Controller tests** (integration):
|
|
|
|
```typescript
|
|
describe("KnowledgeController (e2e)", () => {
|
|
let app: INestApplication;
|
|
let prisma: PrismaService;
|
|
|
|
beforeAll(async () => {
|
|
const module = await Test.createTestingModule({
|
|
imports: [AppModule],
|
|
}).compile();
|
|
|
|
app = module.createNestApplication();
|
|
await app.init();
|
|
|
|
prisma = module.get(PrismaService);
|
|
});
|
|
|
|
afterAll(async () => {
|
|
await prisma.$disconnect();
|
|
await app.close();
|
|
});
|
|
|
|
describe("POST /knowledge/entries", () => {
|
|
it("should create entry and return 201", () => {
|
|
return request(app.getHttpServer())
|
|
.post("/knowledge/entries")
|
|
.set("Authorization", `Bearer ${authToken}`)
|
|
.set("x-workspace-id", workspaceId)
|
|
.send({
|
|
title: "Test Entry",
|
|
content: "# Test Content",
|
|
status: "DRAFT",
|
|
})
|
|
.expect(201)
|
|
.expect((res) => {
|
|
expect(res.body.slug).toBe("test-entry");
|
|
expect(res.body.status).toBe("DRAFT");
|
|
});
|
|
});
|
|
});
|
|
});
|
|
```
|
|
|
|
**Utility tests:**
|
|
|
|
````typescript
|
|
describe("parseWikiLinks", () => {
|
|
it("should parse simple wiki link", () => {
|
|
const content = "See [[My Page]] for details.";
|
|
const links = parseWikiLinks(content);
|
|
|
|
expect(links).toHaveLength(1);
|
|
expect(links[0]).toMatchObject({
|
|
target: "My Page",
|
|
displayText: "My Page",
|
|
raw: "[[My Page]]",
|
|
});
|
|
});
|
|
|
|
it("should parse link with custom display text", () => {
|
|
const content = "See [[page-slug|custom text]] here.";
|
|
const links = parseWikiLinks(content);
|
|
|
|
expect(links[0]).toMatchObject({
|
|
target: "page-slug",
|
|
displayText: "custom text",
|
|
});
|
|
});
|
|
|
|
it("should ignore links in code blocks", () => {
|
|
const content = "```\n[[Not A Link]]\n```";
|
|
const links = parseWikiLinks(content);
|
|
|
|
expect(links).toHaveLength(0);
|
|
});
|
|
});
|
|
````
|
|
|
|
### Test Data Setup
|
|
|
|
Create reusable fixtures:
|
|
|
|
```typescript
|
|
// test/fixtures/knowledge.fixtures.ts
|
|
export const createMockEntry = (overrides = {}) => ({
|
|
id: "entry-uuid",
|
|
workspaceId: "workspace-uuid",
|
|
slug: "test-entry",
|
|
title: "Test Entry",
|
|
content: "# Test",
|
|
contentHtml: "<h1>Test</h1>",
|
|
summary: null,
|
|
status: "DRAFT",
|
|
visibility: "PRIVATE",
|
|
createdAt: new Date(),
|
|
updatedAt: new Date(),
|
|
createdBy: "user-uuid",
|
|
updatedBy: "user-uuid",
|
|
...overrides,
|
|
});
|
|
|
|
export const createMockVersion = (entryId: string, version: number) => ({
|
|
id: `version-${version}-uuid`,
|
|
entryId,
|
|
version,
|
|
title: "Test Entry",
|
|
content: `# Version ${version}`,
|
|
summary: null,
|
|
changeNote: `Update ${version}`,
|
|
createdAt: new Date(),
|
|
createdBy: "user-uuid",
|
|
});
|
|
```
|
|
|
|
---
|
|
|
|
## Contributing
|
|
|
|
### Development Workflow
|
|
|
|
1. **Create feature branch**
|
|
|
|
```bash
|
|
git checkout -b feature/your-feature develop
|
|
```
|
|
|
|
2. **Write tests first** (TDD approach)
|
|
|
|
```bash
|
|
pnpm test:watch knowledge
|
|
```
|
|
|
|
3. **Implement feature**
|
|
- Follow TypeScript strict mode
|
|
- Use existing patterns
|
|
- Add JSDoc comments
|
|
|
|
4. **Run tests and linting**
|
|
|
|
```bash
|
|
pnpm test knowledge
|
|
pnpm lint
|
|
pnpm format
|
|
```
|
|
|
|
5. **Commit with conventional format**
|
|
|
|
```bash
|
|
git commit -m "feat(knowledge): add semantic search endpoint"
|
|
```
|
|
|
|
6. **Create pull request to `develop`**
|
|
|
|
### Code Style
|
|
|
|
**TypeScript:**
|
|
|
|
- Strict mode enabled
|
|
- No `any` types
|
|
- Explicit return types
|
|
- Interface over type when possible
|
|
|
|
**NestJS conventions:**
|
|
|
|
- Services are `@Injectable()`
|
|
- Controllers use `@Controller()`, `@Get()`, etc.
|
|
- DTOs with class-validator decorators
|
|
- Dependency injection via constructor
|
|
|
|
**Naming:**
|
|
|
|
- `camelCase` for variables and functions
|
|
- `PascalCase` for classes and interfaces
|
|
- `UPPER_SNAKE_CASE` for constants
|
|
- `kebab-case` for file names
|
|
|
|
### Adding New Features
|
|
|
|
**New endpoint:**
|
|
|
|
1. Create DTO in `dto/`
|
|
2. Add controller method with proper guards
|
|
3. Implement service method
|
|
4. Write tests (unit + integration)
|
|
5. Update API documentation
|
|
|
|
**New service:**
|
|
|
|
1. Create service class in `services/`
|
|
2. Add `@Injectable()` decorator
|
|
3. Register in `knowledge.module.ts`
|
|
4. Write comprehensive tests
|
|
5. Document public API with JSDoc
|
|
|
|
**Database changes:**
|
|
|
|
1. Update `schema.prisma`
|
|
2. Create migration: `pnpm prisma migrate dev --name your_migration_name`
|
|
3. Update entity interfaces in `entities/`
|
|
4. Update services to use new schema
|
|
5. Write migration tests
|
|
|
|
### Performance Considerations
|
|
|
|
**Always consider:**
|
|
|
|
- Database query efficiency (use indexes)
|
|
- N+1 query problems (use `include` wisely)
|
|
- Cache invalidation strategy
|
|
- Transaction boundaries
|
|
- Large content handling
|
|
|
|
**Optimization checklist:**
|
|
|
|
- [ ] Proper indexes on database columns
|
|
- [ ] Caching for expensive operations
|
|
- [ ] Pagination for list endpoints
|
|
- [ ] Lazy loading for relations
|
|
- [ ] Bulk operations where possible
|
|
|
|
### Security Checklist
|
|
|
|
- [ ] Input validation (DTOs)
|
|
- [ ] Permission guards on endpoints
|
|
- [ ] Workspace isolation (never cross workspaces)
|
|
- [ ] SQL injection prevention (Prisma handles this)
|
|
- [ ] No sensitive data in logs
|
|
- [ ] Rate limiting (future)
|
|
|
|
---
|
|
|
|
## Additional Resources
|
|
|
|
- **[User Guide](KNOWLEDGE_USER_GUIDE.md)** — End-user documentation
|
|
- **[API Documentation](KNOWLEDGE_API.md)** — Complete API reference
|
|
- **[Main README](README.md)** — Project overview
|
|
- **[NestJS Docs](https://docs.nestjs.com/)** — Framework documentation
|
|
- **[Prisma Docs](https://www.prisma.io/docs)** — ORM documentation
|
|
|
|
---
|
|
|
|
**Happy coding! 🚀**
|