Issues resolved: - #68: pgvector Setup * Added pgvector vector index migration for knowledge_embeddings * Vector index uses HNSW algorithm with cosine distance * Optimized for 1536-dimension OpenAI embeddings - #69: Embedding Generation Pipeline * Created EmbeddingService with OpenAI integration * Automatic embedding generation on entry create/update * Batch processing endpoint for existing entries * Async generation to avoid blocking API responses * Content preparation with title weighting - #70: Semantic Search API * POST /api/knowledge/search/semantic - pure vector search * POST /api/knowledge/search/hybrid - RRF combined search * POST /api/knowledge/embeddings/batch - batch generation * Comprehensive test coverage * Full documentation in docs/SEMANTIC_SEARCH.md Technical details: - Uses OpenAI text-embedding-3-small model (1536 dims) - HNSW index for O(log n) similarity search - Reciprocal Rank Fusion for hybrid search - Graceful degradation when OpenAI not configured - Async embedding generation for performance Configuration: - Added OPENAI_API_KEY to .env.example - Optional feature - disabled if API key not set - Falls back to keyword search in hybrid mode
This commit is contained in:
@@ -1,9 +1,10 @@
|
||||
import { Controller, Get, Query, UseGuards } from "@nestjs/common";
|
||||
import { Controller, Get, Post, Body, Query, UseGuards } from "@nestjs/common";
|
||||
import { SearchService, PaginatedSearchResults } from "./services/search.service";
|
||||
import { SearchQueryDto, TagSearchDto, RecentEntriesDto } from "./dto";
|
||||
import { AuthGuard } from "../auth/guards/auth.guard";
|
||||
import { WorkspaceGuard, PermissionGuard } from "../common/guards";
|
||||
import { Workspace, Permission, RequirePermission } from "../common/decorators";
|
||||
import { EntryStatus } from "@prisma/client";
|
||||
import type {
|
||||
PaginatedEntries,
|
||||
KnowledgeEntryWithTags,
|
||||
@@ -97,4 +98,55 @@ export class SearchController {
|
||||
count: entries.length,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /api/knowledge/search/semantic
|
||||
* Semantic search using vector similarity
|
||||
* Requires: Any workspace member, OpenAI API key configured
|
||||
*
|
||||
* @body query - The search query string (required)
|
||||
* @body status - Filter by entry status (optional)
|
||||
* @query page - Page number (default: 1)
|
||||
* @query limit - Results per page (default: 20, max: 100)
|
||||
*/
|
||||
@Post("semantic")
|
||||
@RequirePermission(Permission.WORKSPACE_ANY)
|
||||
async semanticSearch(
|
||||
@Workspace() workspaceId: string,
|
||||
@Body() body: { query: string; status?: EntryStatus },
|
||||
@Query("page") page?: number,
|
||||
@Query("limit") limit?: number
|
||||
): Promise<PaginatedSearchResults> {
|
||||
return this.searchService.semanticSearch(body.query, workspaceId, {
|
||||
status: body.status,
|
||||
page,
|
||||
limit,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /api/knowledge/search/hybrid
|
||||
* Hybrid search combining vector similarity and full-text search
|
||||
* Uses Reciprocal Rank Fusion to merge results
|
||||
* Requires: Any workspace member
|
||||
*
|
||||
* @body query - The search query string (required)
|
||||
* @body status - Filter by entry status (optional)
|
||||
* @query page - Page number (default: 1)
|
||||
* @query limit - Results per page (default: 20, max: 100)
|
||||
*/
|
||||
@Post("hybrid")
|
||||
@RequirePermission(Permission.WORKSPACE_ANY)
|
||||
async hybridSearch(
|
||||
@Workspace() workspaceId: string,
|
||||
@Body() body: { query: string; status?: EntryStatus },
|
||||
@Query("page") page?: number,
|
||||
@Query("limit") limit?: number
|
||||
): Promise<PaginatedSearchResults> {
|
||||
return this.searchService.hybridSearch(body.query, workspaceId, {
|
||||
status: body.status,
|
||||
page,
|
||||
limit,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user