Issues resolved: - #68: pgvector Setup * Added pgvector vector index migration for knowledge_embeddings * Vector index uses HNSW algorithm with cosine distance * Optimized for 1536-dimension OpenAI embeddings - #69: Embedding Generation Pipeline * Created EmbeddingService with OpenAI integration * Automatic embedding generation on entry create/update * Batch processing endpoint for existing entries * Async generation to avoid blocking API responses * Content preparation with title weighting - #70: Semantic Search API * POST /api/knowledge/search/semantic - pure vector search * POST /api/knowledge/search/hybrid - RRF combined search * POST /api/knowledge/embeddings/batch - batch generation * Comprehensive test coverage * Full documentation in docs/SEMANTIC_SEARCH.md Technical details: - Uses OpenAI text-embedding-3-small model (1536 dims) - HNSW index for O(log n) similarity search - Reciprocal Rank Fusion for hybrid search - Graceful degradation when OpenAI not configured - Async embedding generation for performance Configuration: - Added OPENAI_API_KEY to .env.example - Optional feature - disabled if API key not set - Falls back to keyword search in hybrid mode
This commit is contained in:
@@ -12,6 +12,7 @@ import {
|
||||
DefaultValuePipe,
|
||||
} from "@nestjs/common";
|
||||
import type { AuthUser } from "@mosaic/shared";
|
||||
import { EntryStatus } from "@prisma/client";
|
||||
import { KnowledgeService } from "./knowledge.service";
|
||||
import { CreateEntryDto, UpdateEntryDto, EntryQueryDto, RestoreVersionDto } from "./dto";
|
||||
import { AuthGuard } from "../auth/guards/auth.guard";
|
||||
@@ -192,6 +193,38 @@ export class KnowledgeController {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Controller for knowledge embeddings endpoints
|
||||
*/
|
||||
@Controller("knowledge/embeddings")
|
||||
@UseGuards(AuthGuard, WorkspaceGuard, PermissionGuard)
|
||||
export class KnowledgeEmbeddingsController {
|
||||
constructor(private readonly knowledgeService: KnowledgeService) {}
|
||||
|
||||
/**
|
||||
* POST /api/knowledge/embeddings/batch
|
||||
* Batch generate embeddings for all entries in the workspace
|
||||
* Useful for populating embeddings for existing entries
|
||||
* Requires: ADMIN role or higher
|
||||
*/
|
||||
@Post("batch")
|
||||
@RequirePermission(Permission.WORKSPACE_ADMIN)
|
||||
async batchGenerate(
|
||||
@Workspace() workspaceId: string,
|
||||
@Body() body: { status?: string }
|
||||
) {
|
||||
const status = body.status as EntryStatus | undefined;
|
||||
const result = await this.knowledgeService.batchGenerateEmbeddings(
|
||||
workspaceId,
|
||||
status
|
||||
);
|
||||
return {
|
||||
message: `Generated ${result.success} embeddings out of ${result.total} entries`,
|
||||
...result,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Controller for knowledge cache endpoints
|
||||
*/
|
||||
|
||||
Reference in New Issue
Block a user