stack/docs/scratchpads/65-full-text-search.md

Issue #65: [KNOW-013] Full-Text Search Setup

Objective

Set up PostgreSQL full-text search for entries in the knowledge module with weighted fields and proper indexing.

Approach

1. Examine current Prisma schema for knowledge entries
2. Write tests for full-text search functionality (TDD)
3. Add tsvector column to knowledge entries table
4. Create GIN index for performance
5. Implement trigger to maintain tsvector on insert/update
6. Weight fields: title (A), summary (B), content (C)
7. Verify with sample queries

Progress

• Create scratchpad
• Read Prisma schema
• Examine existing search service
• Write failing tests for tsvector column (RED)
• Create migration with tsvector column, GIN index, and triggers
• Update Prisma schema to include tsvector
• Update search service to use precomputed tsvector (GREEN)
• Run tests and verify coverage (8/8 integration tests pass, 205/225 knowledge module tests pass)
• Run quality checks (typecheck and lint pass)
• Commit changes

Current State

The search service already implements full-text search using to_tsvector and ts_rank in raw SQL queries, but it calculates tsvector on-the-fly. This is inefficient for large datasets. The migration will:

1. Add a search_vector tsvector column to knowledge_entries
2. Create a GIN index on search_vector for fast lookups
3. Add a trigger to automatically update search_vector on insert/update
4. Use weighted fields: title (A), summary (B), content (C)

Testing

• Unit tests for search query generation
• Integration tests with actual database queries
• Performance verification with sample data

Notes

• Using PostgreSQL's built-in full-text search capabilities
• GIN index for fast text search
• Automatic maintenance via triggers
• Field weights: A (title) > B (summary) > C (content)